š÷‰ ¾¢ B AssignVariableOp resource value"dtype" dtypetypeˆ 8 Const output"dtype" valuetensor" dtypetype  NoOp C Placeholder output"dtype" dtypetype" shapeshape: @ ReadVariableOp resource value"dtype" dtypetypeˆ ¾ StatefulPartitionedCall args2Tin output2Tout" Tin list(type)(" Tout list(type)(" ffunc" configstring" config_protostring" executor_typestringˆ – VarHandleOp resource" containerstring" shared_namestring" dtypetype" shapeshape"# allowed_devices list(string) ˆ 9 VarIsInitializedOp resource is_initialized ˆ"serve*2.2.02unknown8©Ćš W asset_path_initializer Placeholder* _output_shapes :* dtype0* shape:  Variable VarHandleOp* _class  loc:@Variable* _output_shapes :* dtype0* shape:* shared_name Variable a )Variable/IsInitialized/VarIsInitializedOpVarIsInitializedOpVariable* _output_shapes : R Variable/AssignAssignVariableOpVariableasset_path_initializer* dtype0 ] Variable/Read/ReadVariableOpReadVariableOpVariable* _output_shapes :* dtype0 h Variable_1 VarHandleOp* _output_shapes :* dtype0 * shape:* shared_name  Variable_1 a Variable_1/Read/ReadVariableOpReadVariableOp Variable_1* _output_shapes :* dtype0 “ word_embeddings/embeddings VarHandleOp* _output_shapes :* dtype0* shape : ”Ė€*+ shared_nameword_embeddings/embeddings Œ .word_embeddings/embeddings/Read/ReadVariableOpReadVariableOpword_embeddings/embeddings*! _output_shapes : ”Ė€* dtype0 ˜ position_embedding/embeddings VarHandleOp* _output_shapes :* dtype0* shape : €€*. shared_nameposition_embedding/embeddings ‘ 1position_embedding/embeddings/Read/ReadVariableOpReadVariableOpposition_embedding/embeddings* _output_shapes : €€* dtype0 ‘ type_embeddings/embeddings VarHandleOp* _output_shapes :* dtype0* shape : €*+ shared_nametype_embeddings/embeddings Š .type_embeddings/embeddings/Read/ReadVariableOpReadVariableOptype_embeddings/embeddings* _output_shapes : €* dtype0  embeddings/layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*, shared_nameembeddings/layer_norm/gamma ˆ /embeddings/layer_norm/gamma/Read/ReadVariableOpReadVariableOpembeddings/layer_norm/gamma* _output_shapes :€* dtype0  embeddings/layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*+ shared_nameembeddings/layer_norm/beta † .embeddings/layer_norm/beta/Read/ReadVariableOpReadVariableOpembeddings/layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_0/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_0/self_attention/query/kernel ø Ctransformer/layer_0/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_0/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_0/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_0/self_attention/query/bias Æ Atransformer/layer_0/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_0/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_0/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_0/self_attention/key/kernel “ Atransformer/layer_0/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_0/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_0/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_0/self_attention/key/bias « ?transformer/layer_0/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_0/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_0/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_0/self_attention/value/kernel ø Ctransformer/layer_0/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_0/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_0/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_0/self_attention/value/bias Æ Atransformer/layer_0/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_0/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_0/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_0/self_attention/attention_output/kernel Ī Ntransformer/layer_0/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_0/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_0/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_0/self_attention/attention_output/bias Ā Ltransformer/layer_0/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_0/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_0/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_0/self_attention_layer_norm/gamma ø Gtransformer/layer_0/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_0/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_0/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_0/self_attention_layer_norm/beta ¶ Ftransformer/layer_0/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_0/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_0/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_0/intermediate/kernel „ ;transformer/layer_0/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_0/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_0/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_0/intermediate/bias œ 9transformer/layer_0/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_0/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_0/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_0/output/kernel ™ 5transformer/layer_0/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_0/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_0/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_0/output/bias  3transformer/layer_0/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_0/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_0/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_0/output_layer_norm/gamma Ø ?transformer/layer_0/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_0/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_0/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_0/output_layer_norm/beta ¦ >transformer/layer_0/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_0/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_1/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_1/self_attention/query/kernel ø Ctransformer/layer_1/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_1/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_1/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_1/self_attention/query/bias Æ Atransformer/layer_1/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_1/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_1/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_1/self_attention/key/kernel “ Atransformer/layer_1/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_1/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_1/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_1/self_attention/key/bias « ?transformer/layer_1/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_1/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_1/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_1/self_attention/value/kernel ø Ctransformer/layer_1/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_1/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_1/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_1/self_attention/value/bias Æ Atransformer/layer_1/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_1/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_1/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_1/self_attention/attention_output/kernel Ī Ntransformer/layer_1/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_1/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_1/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_1/self_attention/attention_output/bias Ā Ltransformer/layer_1/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_1/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_1/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_1/self_attention_layer_norm/gamma ø Gtransformer/layer_1/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_1/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_1/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_1/self_attention_layer_norm/beta ¶ Ftransformer/layer_1/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_1/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_1/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_1/intermediate/kernel „ ;transformer/layer_1/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_1/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_1/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_1/intermediate/bias œ 9transformer/layer_1/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_1/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_1/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_1/output/kernel ™ 5transformer/layer_1/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_1/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_1/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_1/output/bias  3transformer/layer_1/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_1/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_1/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_1/output_layer_norm/gamma Ø ?transformer/layer_1/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_1/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_1/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_1/output_layer_norm/beta ¦ >transformer/layer_1/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_1/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_2/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_2/self_attention/query/kernel ø Ctransformer/layer_2/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_2/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_2/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_2/self_attention/query/bias Æ Atransformer/layer_2/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_2/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_2/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_2/self_attention/key/kernel “ Atransformer/layer_2/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_2/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_2/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_2/self_attention/key/bias « ?transformer/layer_2/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_2/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_2/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_2/self_attention/value/kernel ø Ctransformer/layer_2/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_2/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_2/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_2/self_attention/value/bias Æ Atransformer/layer_2/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_2/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_2/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_2/self_attention/attention_output/kernel Ī Ntransformer/layer_2/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_2/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_2/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_2/self_attention/attention_output/bias Ā Ltransformer/layer_2/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_2/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_2/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_2/self_attention_layer_norm/gamma ø Gtransformer/layer_2/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_2/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_2/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_2/self_attention_layer_norm/beta ¶ Ftransformer/layer_2/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_2/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_2/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_2/intermediate/kernel „ ;transformer/layer_2/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_2/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_2/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_2/intermediate/bias œ 9transformer/layer_2/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_2/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_2/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_2/output/kernel ™ 5transformer/layer_2/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_2/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_2/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_2/output/bias  3transformer/layer_2/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_2/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_2/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_2/output_layer_norm/gamma Ø ?transformer/layer_2/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_2/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_2/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_2/output_layer_norm/beta ¦ >transformer/layer_2/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_2/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_3/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_3/self_attention/query/kernel ø Ctransformer/layer_3/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_3/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_3/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_3/self_attention/query/bias Æ Atransformer/layer_3/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_3/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_3/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_3/self_attention/key/kernel “ Atransformer/layer_3/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_3/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_3/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_3/self_attention/key/bias « ?transformer/layer_3/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_3/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_3/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_3/self_attention/value/kernel ø Ctransformer/layer_3/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_3/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_3/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_3/self_attention/value/bias Æ Atransformer/layer_3/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_3/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_3/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_3/self_attention/attention_output/kernel Ī Ntransformer/layer_3/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_3/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_3/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_3/self_attention/attention_output/bias Ā Ltransformer/layer_3/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_3/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_3/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_3/self_attention_layer_norm/gamma ø Gtransformer/layer_3/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_3/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_3/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_3/self_attention_layer_norm/beta ¶ Ftransformer/layer_3/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_3/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_3/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_3/intermediate/kernel „ ;transformer/layer_3/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_3/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_3/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_3/intermediate/bias œ 9transformer/layer_3/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_3/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_3/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_3/output/kernel ™ 5transformer/layer_3/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_3/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_3/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_3/output/bias  3transformer/layer_3/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_3/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_3/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_3/output_layer_norm/gamma Ø ?transformer/layer_3/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_3/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_3/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_3/output_layer_norm/beta ¦ >transformer/layer_3/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_3/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_4/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_4/self_attention/query/kernel ø Ctransformer/layer_4/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_4/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_4/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_4/self_attention/query/bias Æ Atransformer/layer_4/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_4/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_4/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_4/self_attention/key/kernel “ Atransformer/layer_4/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_4/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_4/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_4/self_attention/key/bias « ?transformer/layer_4/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_4/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_4/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_4/self_attention/value/kernel ø Ctransformer/layer_4/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_4/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_4/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_4/self_attention/value/bias Æ Atransformer/layer_4/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_4/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_4/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_4/self_attention/attention_output/kernel Ī Ntransformer/layer_4/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_4/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_4/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_4/self_attention/attention_output/bias Ā Ltransformer/layer_4/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_4/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_4/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_4/self_attention_layer_norm/gamma ø Gtransformer/layer_4/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_4/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_4/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_4/self_attention_layer_norm/beta ¶ Ftransformer/layer_4/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_4/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_4/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_4/intermediate/kernel „ ;transformer/layer_4/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_4/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_4/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_4/intermediate/bias œ 9transformer/layer_4/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_4/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_4/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_4/output/kernel ™ 5transformer/layer_4/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_4/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_4/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_4/output/bias  3transformer/layer_4/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_4/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_4/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_4/output_layer_norm/gamma Ø ?transformer/layer_4/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_4/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_4/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_4/output_layer_norm/beta ¦ >transformer/layer_4/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_4/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_5/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_5/self_attention/query/kernel ø Ctransformer/layer_5/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_5/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_5/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_5/self_attention/query/bias Æ Atransformer/layer_5/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_5/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_5/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_5/self_attention/key/kernel “ Atransformer/layer_5/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_5/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_5/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_5/self_attention/key/bias « ?transformer/layer_5/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_5/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_5/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_5/self_attention/value/kernel ø Ctransformer/layer_5/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_5/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_5/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_5/self_attention/value/bias Æ Atransformer/layer_5/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_5/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_5/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_5/self_attention/attention_output/kernel Ī Ntransformer/layer_5/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_5/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_5/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_5/self_attention/attention_output/bias Ā Ltransformer/layer_5/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_5/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_5/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_5/self_attention_layer_norm/gamma ø Gtransformer/layer_5/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_5/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_5/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_5/self_attention_layer_norm/beta ¶ Ftransformer/layer_5/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_5/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_5/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_5/intermediate/kernel „ ;transformer/layer_5/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_5/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_5/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_5/intermediate/bias œ 9transformer/layer_5/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_5/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_5/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_5/output/kernel ™ 5transformer/layer_5/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_5/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_5/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_5/output/bias  3transformer/layer_5/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_5/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_5/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_5/output_layer_norm/gamma Ø ?transformer/layer_5/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_5/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_5/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_5/output_layer_norm/beta ¦ >transformer/layer_5/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_5/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_6/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_6/self_attention/query/kernel ø Ctransformer/layer_6/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_6/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_6/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_6/self_attention/query/bias Æ Atransformer/layer_6/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_6/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_6/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_6/self_attention/key/kernel “ Atransformer/layer_6/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_6/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_6/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_6/self_attention/key/bias « ?transformer/layer_6/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_6/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_6/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_6/self_attention/value/kernel ø Ctransformer/layer_6/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_6/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_6/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_6/self_attention/value/bias Æ Atransformer/layer_6/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_6/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_6/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_6/self_attention/attention_output/kernel Ī Ntransformer/layer_6/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_6/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_6/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_6/self_attention/attention_output/bias Ā Ltransformer/layer_6/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_6/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_6/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_6/self_attention_layer_norm/gamma ø Gtransformer/layer_6/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_6/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_6/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_6/self_attention_layer_norm/beta ¶ Ftransformer/layer_6/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_6/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_6/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_6/intermediate/kernel „ ;transformer/layer_6/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_6/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_6/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_6/intermediate/bias œ 9transformer/layer_6/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_6/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_6/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_6/output/kernel ™ 5transformer/layer_6/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_6/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_6/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_6/output/bias  3transformer/layer_6/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_6/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_6/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_6/output_layer_norm/gamma Ø ?transformer/layer_6/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_6/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_6/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_6/output_layer_norm/beta ¦ >transformer/layer_6/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_6/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_7/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_7/self_attention/query/kernel ø Ctransformer/layer_7/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_7/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_7/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_7/self_attention/query/bias Æ Atransformer/layer_7/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_7/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_7/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_7/self_attention/key/kernel “ Atransformer/layer_7/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_7/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_7/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_7/self_attention/key/bias « ?transformer/layer_7/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_7/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_7/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_7/self_attention/value/kernel ø Ctransformer/layer_7/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_7/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_7/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_7/self_attention/value/bias Æ Atransformer/layer_7/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_7/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_7/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_7/self_attention/attention_output/kernel Ī Ntransformer/layer_7/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_7/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_7/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_7/self_attention/attention_output/bias Ā Ltransformer/layer_7/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_7/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_7/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_7/self_attention_layer_norm/gamma ø Gtransformer/layer_7/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_7/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_7/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_7/self_attention_layer_norm/beta ¶ Ftransformer/layer_7/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_7/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_7/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_7/intermediate/kernel „ ;transformer/layer_7/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_7/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_7/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_7/intermediate/bias œ 9transformer/layer_7/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_7/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_7/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_7/output/kernel ™ 5transformer/layer_7/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_7/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_7/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_7/output/bias  3transformer/layer_7/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_7/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_7/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_7/output_layer_norm/gamma Ø ?transformer/layer_7/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_7/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_7/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_7/output_layer_norm/beta ¦ >transformer/layer_7/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_7/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_8/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_8/self_attention/query/kernel ø Ctransformer/layer_8/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_8/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_8/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_8/self_attention/query/bias Æ Atransformer/layer_8/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_8/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_8/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_8/self_attention/key/kernel “ Atransformer/layer_8/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_8/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_8/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_8/self_attention/key/bias « ?transformer/layer_8/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_8/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_8/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_8/self_attention/value/kernel ø Ctransformer/layer_8/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_8/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_8/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_8/self_attention/value/bias Æ Atransformer/layer_8/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_8/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_8/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_8/self_attention/attention_output/kernel Ī Ntransformer/layer_8/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_8/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_8/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_8/self_attention/attention_output/bias Ā Ltransformer/layer_8/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_8/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_8/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_8/self_attention_layer_norm/gamma ø Gtransformer/layer_8/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_8/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_8/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_8/self_attention_layer_norm/beta ¶ Ftransformer/layer_8/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_8/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_8/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_8/intermediate/kernel „ ;transformer/layer_8/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_8/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_8/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_8/intermediate/bias œ 9transformer/layer_8/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_8/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_8/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_8/output/kernel ™ 5transformer/layer_8/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_8/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_8/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_8/output/bias  3transformer/layer_8/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_8/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_8/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_8/output_layer_norm/gamma Ø ?transformer/layer_8/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_8/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_8/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_8/output_layer_norm/beta ¦ >transformer/layer_8/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_8/output_layer_norm/beta* _output_shapes :€* dtype0 æ /transformer/layer_9/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_9/self_attention/query/kernel ø Ctransformer/layer_9/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_9/self_attention/query/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_9/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_9/self_attention/query/bias Æ Atransformer/layer_9/self_attention/query/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_9/self_attention/query/bias* _output_shapes : @* dtype0 » -transformer/layer_9/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*> shared_name/-transformer/layer_9/self_attention/key/kernel “ Atransformer/layer_9/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp-transformer/layer_9/self_attention/key/kernel*# _output_shapes : € @* dtype0 ² +transformer/layer_9/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*< shared_name-+transformer/layer_9/self_attention/key/bias « ?transformer/layer_9/self_attention/key/bias/Read/ReadVariableOpReadVariableOp+transformer/layer_9/self_attention/key/bias* _output_shapes : @* dtype0 æ /transformer/layer_9/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*@ shared_name1/transformer/layer_9/self_attention/value/kernel ø Ctransformer/layer_9/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp/transformer/layer_9/self_attention/value/kernel*# _output_shapes : € @* dtype0 ¶ -transformer/layer_9/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*> shared_name/-transformer/layer_9/self_attention/value/bias Æ Atransformer/layer_9/self_attention/value/bias/Read/ReadVariableOpReadVariableOp-transformer/layer_9/self_attention/value/bias* _output_shapes : @* dtype0 Õ :transformer/layer_9/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*K shared_name<:transformer/layer_9/self_attention/attention_output/kernel Ī Ntransformer/layer_9/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp:transformer/layer_9/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 É 8transformer/layer_9/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*I shared_name:8transformer/layer_9/self_attention/attention_output/bias Ā Ltransformer/layer_9/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp8transformer/layer_9/self_attention/attention_output/bias* _output_shapes :€* dtype0 æ 3transformer/layer_9/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_9/self_attention_layer_norm/gamma ø Gtransformer/layer_9/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp3transformer/layer_9/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 ½ 2transformer/layer_9/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*C shared_name42transformer/layer_9/self_attention_layer_norm/beta ¶ Ftransformer/layer_9/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp2transformer/layer_9/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ¬ 'transformer/layer_9/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*8 shared_name)'transformer/layer_9/intermediate/kernel „ ;transformer/layer_9/intermediate/kernel/Read/ReadVariableOpReadVariableOp'transformer/layer_9/intermediate/kernel* _output_shapes : €€* dtype0 £ %transformer/layer_9/intermediate/bias VarHandleOp* _output_shapes :* dtype0* shape:€*6 shared_name'%transformer/layer_9/intermediate/bias œ 9transformer/layer_9/intermediate/bias/Read/ReadVariableOpReadVariableOp%transformer/layer_9/intermediate/bias* _output_shapes :€* dtype0   !transformer/layer_9/output/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*2 shared_name#!transformer/layer_9/output/kernel ™ 5transformer/layer_9/output/kernel/Read/ReadVariableOpReadVariableOp!transformer/layer_9/output/kernel* _output_shapes : €€* dtype0 — transformer/layer_9/output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*0 shared_name!transformer/layer_9/output/bias  3transformer/layer_9/output/bias/Read/ReadVariableOpReadVariableOptransformer/layer_9/output/bias* _output_shapes :€* dtype0 Æ +transformer/layer_9/output_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*< shared_name-+transformer/layer_9/output_layer_norm/gamma Ø ?transformer/layer_9/output_layer_norm/gamma/Read/ReadVariableOpReadVariableOp+transformer/layer_9/output_layer_norm/gamma* _output_shapes :€* dtype0 ­ *transformer/layer_9/output_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*; shared_name,*transformer/layer_9/output_layer_norm/beta ¦ >transformer/layer_9/output_layer_norm/beta/Read/ReadVariableOpReadVariableOp*transformer/layer_9/output_layer_norm/beta* _output_shapes :€* dtype0 Į 0transformer/layer_10/self_attention/query/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*A shared_name20transformer/layer_10/self_attention/query/kernel ŗ Dtransformer/layer_10/self_attention/query/kernel/Read/ReadVariableOpReadVariableOp0transformer/layer_10/self_attention/query/kernel*# _output_shapes : € @* dtype0 ø .transformer/layer_10/self_attention/query/bias VarHandleOp* _output_shapes :* dtype0* shape : @*? shared_name0.transformer/layer_10/self_attention/query/bias ± Btransformer/layer_10/self_attention/query/bias/Read/ReadVariableOpReadVariableOp.transformer/layer_10/self_attention/query/bias* _output_shapes : @* dtype0 ½ .transformer/layer_10/self_attention/key/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*? shared_name0.transformer/layer_10/self_attention/key/kernel ¶ Btransformer/layer_10/self_attention/key/kernel/Read/ReadVariableOpReadVariableOp.transformer/layer_10/self_attention/key/kernel*# _output_shapes : € @* dtype0 “ ,transformer/layer_10/self_attention/key/bias VarHandleOp* _output_shapes :* dtype0* shape : @*= shared_name.,transformer/layer_10/self_attention/key/bias ­ @transformer/layer_10/self_attention/key/bias/Read/ReadVariableOpReadVariableOp,transformer/layer_10/self_attention/key/bias* _output_shapes : @* dtype0 Į 0transformer/layer_10/self_attention/value/kernel VarHandleOp* _output_shapes :* dtype0* shape: € @*A shared_name20transformer/layer_10/self_attention/value/kernel ŗ Dtransformer/layer_10/self_attention/value/kernel/Read/ReadVariableOpReadVariableOp0transformer/layer_10/self_attention/value/kernel*# _output_shapes : € @* dtype0 ø .transformer/layer_10/self_attention/value/bias VarHandleOp* _output_shapes :* dtype0* shape : @*? shared_name0.transformer/layer_10/self_attention/value/bias ± Btransformer/layer_10/self_attention/value/bias/Read/ReadVariableOpReadVariableOp.transformer/layer_10/self_attention/value/bias* _output_shapes : @* dtype0 × ;transformer/layer_10/self_attention/attention_output/kernel VarHandleOp* _output_shapes :* dtype0* shape:  @€*L shared_name=;transformer/layer_10/self_attention/attention_output/kernel Š Otransformer/layer_10/self_attention/attention_output/kernel/Read/ReadVariableOpReadVariableOp;transformer/layer_10/self_attention/attention_output/kernel*# _output_shapes :  @€* dtype0 Ė 9transformer/layer_10/self_attention/attention_output/bias VarHandleOp* _output_shapes :* dtype0* shape:€*J shared_name;9transformer/layer_10/self_attention/attention_output/bias Ä Mtransformer/layer_10/self_attention/attention_output/bias/Read/ReadVariableOpReadVariableOp9transformer/layer_10/self_attention/attention_output/bias* _output_shapes :€* dtype0 Į 4transformer/layer_10/self_attention_layer_norm/gamma VarHandleOp* _output_shapes :* dtype0* shape:€*E shared_name64transformer/layer_10/self_attention_layer_norm/gamma ŗ Htransformer/layer_10/self_attention_layer_norm/gamma/Read/ReadVariableOpReadVariableOp4transformer/layer_10/self_attention_layer_norm/gamma* _output_shapes :€* dtype0 æ 3transformer/layer_10/self_attention_layer_norm/beta VarHandleOp* _output_shapes :* dtype0* shape:€*D shared_name53transformer/layer_10/self_attention_layer_norm/beta ø Gtransformer/layer_10/self_attention_layer_norm/beta/Read/ReadVariableOpReadVariableOp3transformer/layer_10/self_attention_layer_norm/beta* _output_shapes :€* dtype0 ® (transformer/layer_10/intermediate/kernel VarHandleOp* _output_shapes :* dtype0* shape : €€*9 shared_name*(transformer/layer_10/intermediate/kernel § 22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198 199 € (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198 ² ļ layer_metrics  variables šnon_trainable_variables ńlayers regularization_losses ņmetrics ólayer_regularization_losses  trainable_variables f ( embeddings ō variables õregularization_losses ötrainable_variables ÷ keras_api € ) embeddings )_position_embeddings ų variables łregularization_losses śtrainable_variables ū keras_api f * embeddings ü variables żregularization_losses žtrainable_variables ’ keras_api V € variables regularization_losses ‚trainable_variables ƒ keras_api v „axis +gamma ,beta … variables †regularization_losses ‡trainable_variables ˆ keras_api V ‰ variables Šregularization_losses ‹trainable_variables Œ keras_api V  variables Žregularization_losses trainable_variables  keras_api Ā ‘_attention_layer ’_attention_output_dense “_attention_dropout ”_attention_layer_norm •_intermediate_dense #–_intermediate_activation_layer — _output_dense ˜_output_dropout ™_output_layer_norm š variables ›regularization_losses œtrainable_variables  keras_api Ā ž_attention_layer Ÿ_attention_output_dense  _attention_dropout ”_attention_layer_norm ¢_intermediate_dense #£_intermediate_activation_layer ¤ _output_dense „_output_dropout ¦_output_layer_norm § variables Øregularization_losses ©trainable_variables Ŗ keras_api Ā «_attention_layer ¬_attention_output_dense ­_attention_dropout ®_attention_layer_norm Æ_intermediate_dense #°_intermediate_activation_layer ± _output_dense ²_output_dropout ³_output_layer_norm “ variables µregularization_losses ¶trainable_variables · keras_api Ā ø_attention_layer ¹_attention_output_dense ŗ_attention_dropout »_attention_layer_norm ¼_intermediate_dense #½_intermediate_activation_layer ¾ _output_dense æ_output_dropout Ą_output_layer_norm Į variables Āregularization_losses Ćtrainable_variables Ä keras_api Ā Å_attention_layer Ę_attention_output_dense Ē_attention_dropout Č_attention_layer_norm É_intermediate_dense #Ź_intermediate_activation_layer Ė _output_dense Ģ_output_dropout Ķ_output_layer_norm Ī variables Ļregularization_losses Štrainable_variables Ń keras_api Ā Ņ_attention_layer Ó_attention_output_dense Ō_attention_dropout Õ_attention_layer_norm Ö_intermediate_dense #×_intermediate_activation_layer Ų _output_dense Ł_output_dropout Ś_output_layer_norm Ū variables Üregularization_losses Żtrainable_variables Ž keras_api Ā ß_attention_layer ą_attention_output_dense į_attention_dropout ā_attention_layer_norm ć_intermediate_dense #ä_intermediate_activation_layer å _output_dense ę_output_dropout ē_output_layer_norm č variables éregularization_losses źtrainable_variables ė keras_api Ā ģ_attention_layer ķ_attention_output_dense ī_attention_dropout ļ_attention_layer_norm š_intermediate_dense #ń_intermediate_activation_layer ņ _output_dense ó_output_dropout ō_output_layer_norm õ variables öregularization_losses ÷trainable_variables ų keras_api Ā ł_attention_layer ś_attention_output_dense ū_attention_dropout ü_attention_layer_norm ż_intermediate_dense #ž_intermediate_activation_layer ’ _output_dense €_output_dropout _output_layer_norm ‚ variables ƒregularization_losses „trainable_variables … keras_api Ā †_attention_layer ‡_attention_output_dense ˆ_attention_dropout ‰_attention_layer_norm Š_intermediate_dense #‹_intermediate_activation_layer Œ _output_dense _output_dropout Ž_output_layer_norm  variables regularization_losses ‘trainable_variables ’ keras_api Ā “_attention_layer ”_attention_output_dense •_attention_dropout –_attention_layer_norm —_intermediate_dense #˜_intermediate_activation_layer ™ _output_dense š_output_dropout ›_output_layer_norm œ variables regularization_losses žtrainable_variables Ÿ keras_api Ā  _attention_layer ”_attention_output_dense ¢_attention_dropout £_attention_layer_norm ¤_intermediate_dense #„_intermediate_activation_layer ¦ _output_dense §_output_dropout Ø_output_layer_norm © variables Ŗregularization_losses «trainable_variables ¬ keras_api V ­ variables ®regularization_losses Ætrainable_variables ° keras_api n ķkernel ībias ± variables ²regularization_losses ³trainable_variables “ keras_api € (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198 € (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198 ² µ layer_metrics $ variables ¶non_trainable_variables ·layers %regularization_losses ømetrics ¹layer_regularization_losses &trainable_variables VT VARIABLE_VALUEword_embeddings/embeddings&variables/0/.ATTRIBUTES/VARIABLE_VALUE YW VARIABLE_VALUEposition_embedding/embeddings&variables/1/.ATTRIBUTES/VARIABLE_VALUE VT VARIABLE_VALUEtype_embeddings/embeddings&variables/2/.ATTRIBUTES/VARIABLE_VALUE WU VARIABLE_VALUEembeddings/layer_norm/gamma&variables/3/.ATTRIBUTES/VARIABLE_VALUE VT VARIABLE_VALUEembeddings/layer_norm/beta&variables/4/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE/transformer/layer_0/self_attention/query/kernel&variables/5/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE-transformer/layer_0/self_attention/query/bias&variables/6/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE-transformer/layer_0/self_attention/key/kernel&variables/7/.ATTRIBUTES/VARIABLE_VALUE ge VARIABLE_VALUE+transformer/layer_0/self_attention/key/bias&variables/8/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE/transformer/layer_0/self_attention/value/kernel&variables/9/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_0/self_attention/value/bias'variables/10/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE:transformer/layer_0/self_attention/attention_output/kernel'variables/11/.ATTRIBUTES/VARIABLE_VALUE us VARIABLE_VALUE8transformer/layer_0/self_attention/attention_output/bias'variables/12/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE3transformer/layer_0/self_attention_layer_norm/gamma'variables/13/.ATTRIBUTES/VARIABLE_VALUE om VARIABLE_VALUE2transformer/layer_0/self_attention_layer_norm/beta'variables/14/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE'transformer/layer_0/intermediate/kernel'variables/15/.ATTRIBUTES/VARIABLE_VALUE b` VARIABLE_VALUE%transformer/layer_0/intermediate/bias'variables/16/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE!transformer/layer_0/output/kernel'variables/17/.ATTRIBUTES/VARIABLE_VALUE \Z VARIABLE_VALUEtransformer/layer_0/output/bias'variables/18/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_0/output_layer_norm/gamma'variables/19/.ATTRIBUTES/VARIABLE_VALUE ge VARIABLE_VALUE*transformer/layer_0/output_layer_norm/beta'variables/20/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_1/self_attention/query/kernel'variables/21/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_1/self_attention/query/bias'variables/22/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_1/self_attention/key/kernel'variables/23/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_1/self_attention/key/bias'variables/24/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_1/self_attention/value/kernel'variables/25/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_1/self_attention/value/bias'variables/26/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE:transformer/layer_1/self_attention/attention_output/kernel'variables/27/.ATTRIBUTES/VARIABLE_VALUE us VARIABLE_VALUE8transformer/layer_1/self_attention/attention_output/bias'variables/28/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE3transformer/layer_1/self_attention_layer_norm/gamma'variables/29/.ATTRIBUTES/VARIABLE_VALUE om VARIABLE_VALUE2transformer/layer_1/self_attention_layer_norm/beta'variables/30/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE'transformer/layer_1/intermediate/kernel'variables/31/.ATTRIBUTES/VARIABLE_VALUE b` VARIABLE_VALUE%transformer/layer_1/intermediate/bias'variables/32/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE!transformer/layer_1/output/kernel'variables/33/.ATTRIBUTES/VARIABLE_VALUE \Z VARIABLE_VALUEtransformer/layer_1/output/bias'variables/34/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_1/output_layer_norm/gamma'variables/35/.ATTRIBUTES/VARIABLE_VALUE ge VARIABLE_VALUE*transformer/layer_1/output_layer_norm/beta'variables/36/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_2/self_attention/query/kernel'variables/37/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_2/self_attention/query/bias'variables/38/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_2/self_attention/key/kernel'variables/39/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_2/self_attention/key/bias'variables/40/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_2/self_attention/value/kernel'variables/41/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_2/self_attention/value/bias'variables/42/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE:transformer/layer_2/self_attention/attention_output/kernel'variables/43/.ATTRIBUTES/VARIABLE_VALUE us VARIABLE_VALUE8transformer/layer_2/self_attention/attention_output/bias'variables/44/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE3transformer/layer_2/self_attention_layer_norm/gamma'variables/45/.ATTRIBUTES/VARIABLE_VALUE om VARIABLE_VALUE2transformer/layer_2/self_attention_layer_norm/beta'variables/46/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE'transformer/layer_2/intermediate/kernel'variables/47/.ATTRIBUTES/VARIABLE_VALUE b` VARIABLE_VALUE%transformer/layer_2/intermediate/bias'variables/48/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE!transformer/layer_2/output/kernel'variables/49/.ATTRIBUTES/VARIABLE_VALUE \Z VARIABLE_VALUEtransformer/layer_2/output/bias'variables/50/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_2/output_layer_norm/gamma'variables/51/.ATTRIBUTES/VARIABLE_VALUE ge VARIABLE_VALUE*transformer/layer_2/output_layer_norm/beta'variables/52/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_3/self_attention/query/kernel'variables/53/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_3/self_attention/query/bias'variables/54/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_3/self_attention/key/kernel'variables/55/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_3/self_attention/key/bias'variables/56/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_3/self_attention/value/kernel'variables/57/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_3/self_attention/value/bias'variables/58/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE:transformer/layer_3/self_attention/attention_output/kernel'variables/59/.ATTRIBUTES/VARIABLE_VALUE us VARIABLE_VALUE8transformer/layer_3/self_attention/attention_output/bias'variables/60/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE3transformer/layer_3/self_attention_layer_norm/gamma'variables/61/.ATTRIBUTES/VARIABLE_VALUE om VARIABLE_VALUE2transformer/layer_3/self_attention_layer_norm/beta'variables/62/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE'transformer/layer_3/intermediate/kernel'variables/63/.ATTRIBUTES/VARIABLE_VALUE b` VARIABLE_VALUE%transformer/layer_3/intermediate/bias'variables/64/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE!transformer/layer_3/output/kernel'variables/65/.ATTRIBUTES/VARIABLE_VALUE \Z VARIABLE_VALUEtransformer/layer_3/output/bias'variables/66/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_3/output_layer_norm/gamma'variables/67/.ATTRIBUTES/VARIABLE_VALUE ge VARIABLE_VALUE*transformer/layer_3/output_layer_norm/beta'variables/68/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_4/self_attention/query/kernel'variables/69/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_4/self_attention/query/bias'variables/70/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_4/self_attention/key/kernel'variables/71/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_4/self_attention/key/bias'variables/72/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_4/self_attention/value/kernel'variables/73/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_4/self_attention/value/bias'variables/74/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE:transformer/layer_4/self_attention/attention_output/kernel'variables/75/.ATTRIBUTES/VARIABLE_VALUE us VARIABLE_VALUE8transformer/layer_4/self_attention/attention_output/bias'variables/76/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE3transformer/layer_4/self_attention_layer_norm/gamma'variables/77/.ATTRIBUTES/VARIABLE_VALUE om VARIABLE_VALUE2transformer/layer_4/self_attention_layer_norm/beta'variables/78/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE'transformer/layer_4/intermediate/kernel'variables/79/.ATTRIBUTES/VARIABLE_VALUE b` VARIABLE_VALUE%transformer/layer_4/intermediate/bias'variables/80/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE!transformer/layer_4/output/kernel'variables/81/.ATTRIBUTES/VARIABLE_VALUE \Z VARIABLE_VALUEtransformer/layer_4/output/bias'variables/82/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_4/output_layer_norm/gamma'variables/83/.ATTRIBUTES/VARIABLE_VALUE ge VARIABLE_VALUE*transformer/layer_4/output_layer_norm/beta'variables/84/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_5/self_attention/query/kernel'variables/85/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_5/self_attention/query/bias'variables/86/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_5/self_attention/key/kernel'variables/87/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_5/self_attention/key/bias'variables/88/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE/transformer/layer_5/self_attention/value/kernel'variables/89/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE-transformer/layer_5/self_attention/value/bias'variables/90/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE:transformer/layer_5/self_attention/attention_output/kernel'variables/91/.ATTRIBUTES/VARIABLE_VALUE us VARIABLE_VALUE8transformer/layer_5/self_attention/attention_output/bias'variables/92/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE3transformer/layer_5/self_attention_layer_norm/gamma'variables/93/.ATTRIBUTES/VARIABLE_VALUE om VARIABLE_VALUE2transformer/layer_5/self_attention_layer_norm/beta'variables/94/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE'transformer/layer_5/intermediate/kernel'variables/95/.ATTRIBUTES/VARIABLE_VALUE b` VARIABLE_VALUE%transformer/layer_5/intermediate/bias'variables/96/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE!transformer/layer_5/output/kernel'variables/97/.ATTRIBUTES/VARIABLE_VALUE \Z VARIABLE_VALUEtransformer/layer_5/output/bias'variables/98/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE+transformer/layer_5/output_layer_norm/gamma'variables/99/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE*transformer/layer_5/output_layer_norm/beta(variables/100/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_6/self_attention/query/kernel(variables/101/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_6/self_attention/query/bias(variables/102/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_6/self_attention/key/kernel(variables/103/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_6/self_attention/key/bias(variables/104/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_6/self_attention/value/kernel(variables/105/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_6/self_attention/value/bias(variables/106/.ATTRIBUTES/VARIABLE_VALUE xv VARIABLE_VALUE:transformer/layer_6/self_attention/attention_output/kernel(variables/107/.ATTRIBUTES/VARIABLE_VALUE vt VARIABLE_VALUE8transformer/layer_6/self_attention/attention_output/bias(variables/108/.ATTRIBUTES/VARIABLE_VALUE qo VARIABLE_VALUE3transformer/layer_6/self_attention_layer_norm/gamma(variables/109/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE2transformer/layer_6/self_attention_layer_norm/beta(variables/110/.ATTRIBUTES/VARIABLE_VALUE ec VARIABLE_VALUE'transformer/layer_6/intermediate/kernel(variables/111/.ATTRIBUTES/VARIABLE_VALUE ca VARIABLE_VALUE%transformer/layer_6/intermediate/bias(variables/112/.ATTRIBUTES/VARIABLE_VALUE _] VARIABLE_VALUE!transformer/layer_6/output/kernel(variables/113/.ATTRIBUTES/VARIABLE_VALUE ][ VARIABLE_VALUEtransformer/layer_6/output/bias(variables/114/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_6/output_layer_norm/gamma(variables/115/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE*transformer/layer_6/output_layer_norm/beta(variables/116/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_7/self_attention/query/kernel(variables/117/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_7/self_attention/query/bias(variables/118/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_7/self_attention/key/kernel(variables/119/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_7/self_attention/key/bias(variables/120/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_7/self_attention/value/kernel(variables/121/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_7/self_attention/value/bias(variables/122/.ATTRIBUTES/VARIABLE_VALUE xv VARIABLE_VALUE:transformer/layer_7/self_attention/attention_output/kernel(variables/123/.ATTRIBUTES/VARIABLE_VALUE vt VARIABLE_VALUE8transformer/layer_7/self_attention/attention_output/bias(variables/124/.ATTRIBUTES/VARIABLE_VALUE qo VARIABLE_VALUE3transformer/layer_7/self_attention_layer_norm/gamma(variables/125/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE2transformer/layer_7/self_attention_layer_norm/beta(variables/126/.ATTRIBUTES/VARIABLE_VALUE ec VARIABLE_VALUE'transformer/layer_7/intermediate/kernel(variables/127/.ATTRIBUTES/VARIABLE_VALUE ca VARIABLE_VALUE%transformer/layer_7/intermediate/bias(variables/128/.ATTRIBUTES/VARIABLE_VALUE _] VARIABLE_VALUE!transformer/layer_7/output/kernel(variables/129/.ATTRIBUTES/VARIABLE_VALUE ][ VARIABLE_VALUEtransformer/layer_7/output/bias(variables/130/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_7/output_layer_norm/gamma(variables/131/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE*transformer/layer_7/output_layer_norm/beta(variables/132/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_8/self_attention/query/kernel(variables/133/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_8/self_attention/query/bias(variables/134/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_8/self_attention/key/kernel(variables/135/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_8/self_attention/key/bias(variables/136/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_8/self_attention/value/kernel(variables/137/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_8/self_attention/value/bias(variables/138/.ATTRIBUTES/VARIABLE_VALUE xv VARIABLE_VALUE:transformer/layer_8/self_attention/attention_output/kernel(variables/139/.ATTRIBUTES/VARIABLE_VALUE vt VARIABLE_VALUE8transformer/layer_8/self_attention/attention_output/bias(variables/140/.ATTRIBUTES/VARIABLE_VALUE qo VARIABLE_VALUE3transformer/layer_8/self_attention_layer_norm/gamma(variables/141/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE2transformer/layer_8/self_attention_layer_norm/beta(variables/142/.ATTRIBUTES/VARIABLE_VALUE ec VARIABLE_VALUE'transformer/layer_8/intermediate/kernel(variables/143/.ATTRIBUTES/VARIABLE_VALUE ca VARIABLE_VALUE%transformer/layer_8/intermediate/bias(variables/144/.ATTRIBUTES/VARIABLE_VALUE _] VARIABLE_VALUE!transformer/layer_8/output/kernel(variables/145/.ATTRIBUTES/VARIABLE_VALUE ][ VARIABLE_VALUEtransformer/layer_8/output/bias(variables/146/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_8/output_layer_norm/gamma(variables/147/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE*transformer/layer_8/output_layer_norm/beta(variables/148/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_9/self_attention/query/kernel(variables/149/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_9/self_attention/query/bias(variables/150/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_9/self_attention/key/kernel(variables/151/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_9/self_attention/key/bias(variables/152/.ATTRIBUTES/VARIABLE_VALUE mk VARIABLE_VALUE/transformer/layer_9/self_attention/value/kernel(variables/153/.ATTRIBUTES/VARIABLE_VALUE ki VARIABLE_VALUE-transformer/layer_9/self_attention/value/bias(variables/154/.ATTRIBUTES/VARIABLE_VALUE xv VARIABLE_VALUE:transformer/layer_9/self_attention/attention_output/kernel(variables/155/.ATTRIBUTES/VARIABLE_VALUE vt VARIABLE_VALUE8transformer/layer_9/self_attention/attention_output/bias(variables/156/.ATTRIBUTES/VARIABLE_VALUE qo VARIABLE_VALUE3transformer/layer_9/self_attention_layer_norm/gamma(variables/157/.ATTRIBUTES/VARIABLE_VALUE pn VARIABLE_VALUE2transformer/layer_9/self_attention_layer_norm/beta(variables/158/.ATTRIBUTES/VARIABLE_VALUE ec VARIABLE_VALUE'transformer/layer_9/intermediate/kernel(variables/159/.ATTRIBUTES/VARIABLE_VALUE ca VARIABLE_VALUE%transformer/layer_9/intermediate/bias(variables/160/.ATTRIBUTES/VARIABLE_VALUE _] VARIABLE_VALUE!transformer/layer_9/output/kernel(variables/161/.ATTRIBUTES/VARIABLE_VALUE ][ VARIABLE_VALUEtransformer/layer_9/output/bias(variables/162/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_9/output_layer_norm/gamma(variables/163/.ATTRIBUTES/VARIABLE_VALUE hf VARIABLE_VALUE*transformer/layer_9/output_layer_norm/beta(variables/164/.ATTRIBUTES/VARIABLE_VALUE nl VARIABLE_VALUE0transformer/layer_10/self_attention/query/kernel(variables/165/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE.transformer/layer_10/self_attention/query/bias(variables/166/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE.transformer/layer_10/self_attention/key/kernel(variables/167/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE,transformer/layer_10/self_attention/key/bias(variables/168/.ATTRIBUTES/VARIABLE_VALUE nl VARIABLE_VALUE0transformer/layer_10/self_attention/value/kernel(variables/169/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE.transformer/layer_10/self_attention/value/bias(variables/170/.ATTRIBUTES/VARIABLE_VALUE yw VARIABLE_VALUE;transformer/layer_10/self_attention/attention_output/kernel(variables/171/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE9transformer/layer_10/self_attention/attention_output/bias(variables/172/.ATTRIBUTES/VARIABLE_VALUE rp VARIABLE_VALUE4transformer/layer_10/self_attention_layer_norm/gamma(variables/173/.ATTRIBUTES/VARIABLE_VALUE qo VARIABLE_VALUE3transformer/layer_10/self_attention_layer_norm/beta(variables/174/.ATTRIBUTES/VARIABLE_VALUE fd VARIABLE_VALUE(transformer/layer_10/intermediate/kernel(variables/175/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE&transformer/layer_10/intermediate/bias(variables/176/.ATTRIBUTES/VARIABLE_VALUE `^ VARIABLE_VALUE"transformer/layer_10/output/kernel(variables/177/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE transformer/layer_10/output/bias(variables/178/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE,transformer/layer_10/output_layer_norm/gamma(variables/179/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_10/output_layer_norm/beta(variables/180/.ATTRIBUTES/VARIABLE_VALUE nl VARIABLE_VALUE0transformer/layer_11/self_attention/query/kernel(variables/181/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE.transformer/layer_11/self_attention/query/bias(variables/182/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE.transformer/layer_11/self_attention/key/kernel(variables/183/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE,transformer/layer_11/self_attention/key/bias(variables/184/.ATTRIBUTES/VARIABLE_VALUE nl VARIABLE_VALUE0transformer/layer_11/self_attention/value/kernel(variables/185/.ATTRIBUTES/VARIABLE_VALUE lj VARIABLE_VALUE.transformer/layer_11/self_attention/value/bias(variables/186/.ATTRIBUTES/VARIABLE_VALUE yw VARIABLE_VALUE;transformer/layer_11/self_attention/attention_output/kernel(variables/187/.ATTRIBUTES/VARIABLE_VALUE wu VARIABLE_VALUE9transformer/layer_11/self_attention/attention_output/bias(variables/188/.ATTRIBUTES/VARIABLE_VALUE rp VARIABLE_VALUE4transformer/layer_11/self_attention_layer_norm/gamma(variables/189/.ATTRIBUTES/VARIABLE_VALUE qo VARIABLE_VALUE3transformer/layer_11/self_attention_layer_norm/beta(variables/190/.ATTRIBUTES/VARIABLE_VALUE fd VARIABLE_VALUE(transformer/layer_11/intermediate/kernel(variables/191/.ATTRIBUTES/VARIABLE_VALUE db VARIABLE_VALUE&transformer/layer_11/intermediate/bias(variables/192/.ATTRIBUTES/VARIABLE_VALUE `^ VARIABLE_VALUE"transformer/layer_11/output/kernel(variables/193/.ATTRIBUTES/VARIABLE_VALUE ^\ VARIABLE_VALUE transformer/layer_11/output/bias(variables/194/.ATTRIBUTES/VARIABLE_VALUE jh VARIABLE_VALUE,transformer/layer_11/output_layer_norm/gamma(variables/195/.ATTRIBUTES/VARIABLE_VALUE ig VARIABLE_VALUE+transformer/layer_11/output_layer_norm/beta(variables/196/.ATTRIBUTES/VARIABLE_VALUE US VARIABLE_VALUEpooler_transform/kernel(variables/197/.ATTRIBUTES/VARIABLE_VALUE SQ VARIABLE_VALUEpooler_transform/bias(variables/198/.ATTRIBUTES/VARIABLE_VALUE  0  0 1 2 3  (0  (0 µ ŗ layer_metrics ō variables »non_trainable_variables ¼layers õregularization_losses ½metrics ¾layer_regularization_losses ötrainable_variables  )0  )0 µ æ layer_metrics ų variables Ąnon_trainable_variables Įlayers łregularization_losses Āmetrics Ćlayer_regularization_losses śtrainable_variables  *0  *0 µ Ä layer_metrics ü variables Ånon_trainable_variables Ęlayers żregularization_losses Ēmetrics Člayer_regularization_losses žtrainable_variables µ É layer_metrics € variables Źnon_trainable_variables Ėlayers regularization_losses Ģmetrics Ķlayer_regularization_losses ‚trainable_variables  +0 ,1  +0 ,1 µ Ī layer_metrics … variables Ļnon_trainable_variables Šlayers †regularization_losses Ńmetrics Ņlayer_regularization_losses ‡trainable_variables µ Ó layer_metrics ‰ variables Ōnon_trainable_variables Õlayers Šregularization_losses Ömetrics ×layer_regularization_losses ‹trainable_variables µ Ų layer_metrics  variables Łnon_trainable_variables Ślayers Žregularization_losses Ūmetrics Ülayer_regularization_losses trainable_variables Ģ Ż _query_dense Ž _key_dense ß _value_dense ą_masked_softmax į_dropout_layer ’ _output_dense ā variables ćregularization_losses ätrainable_variables å keras_api Ÿ ępartial_output_shape ēfull_output_shape 3kernel 4bias č variables éregularization_losses źtrainable_variables ė keras_api V ģ variables ķregularization_losses ītrainable_variables ļ keras_api v šaxis 5gamma 6beta ń variables ņregularization_losses ótrainable_variables ō keras_api „ 7kernel 7_kernel 8bias 8_bias õ variables öregularization_losses ÷trainable_variables ų keras_api V ł variables śregularization_losses ūtrainable_variables ü keras_api „ 9kernel 9_kernel :bias :_bias ż variables žregularization_losses ’trainable_variables € keras_api V  variables ‚regularization_losses ƒtrainable_variables „ keras_api v …axis ;gamma <beta † variables ‡regularization_losses ˆtrainable_variables ‰ keras_api v -0 .1 /2 03 14 25 36 47 58 69 710 811 912 :13 ;14 <15 v -0 .1 /2 03 14 25 36 47 58 69 710 811 912 :13 ;14 <15 µ Š layer_metrics š variables ‹non_trainable_variables Œlayers ›regularization_losses metrics Žlayer_regularization_losses œtrainable_variables Ģ  _query_dense  _key_dense ‘ _value_dense ’_masked_softmax “_dropout_layer Ÿ _output_dense ” variables •regularization_losses –trainable_variables — keras_api Ÿ ˜partial_output_shape ™full_output_shape Ckernel Dbias š variables ›regularization_losses œtrainable_variables  keras_api V ž variables Ÿregularization_losses  trainable_variables ” keras_api v ¢axis Egamma Fbeta £ variables ¤regularization_losses „trainable_variables ¦ keras_api „ Gkernel G_kernel Hbias H_bias § variables Øregularization_losses ©trainable_variables Ŗ keras_api V « variables ¬regularization_losses ­trainable_variables ® keras_api „ Ikernel I_kernel Jbias J_bias Æ variables °regularization_losses ±trainable_variables ² keras_api V ³ variables “regularization_losses µtrainable_variables ¶ keras_api v ·axis Kgamma Lbeta ø variables ¹regularization_losses ŗtrainable_variables » keras_api v =0 >1 ?2 @3 A4 B5 C6 D7 E8 F9 G10 H11 I12 J13 K14 L15 v =0 >1 ?2 @3 A4 B5 C6 D7 E8 F9 G10 H11 I12 J13 K14 L15 µ ¼ layer_metrics § variables ½non_trainable_variables ¾layers Øregularization_losses æmetrics Ąlayer_regularization_losses ©trainable_variables Ģ Į _query_dense Ā _key_dense Ć _value_dense Ä_masked_softmax Å_dropout_layer ¬ _output_dense Ę variables Ēregularization_losses Čtrainable_variables É keras_api Ÿ Źpartial_output_shape Ėfull_output_shape Skernel Tbias Ģ variables Ķregularization_losses Ītrainable_variables Ļ keras_api V Š variables Ńregularization_losses Ņtrainable_variables Ó keras_api v Ōaxis Ugamma Vbeta Õ variables Öregularization_losses ×trainable_variables Ų keras_api „ Wkernel W_kernel Xbias X_bias Ł variables Śregularization_losses Ūtrainable_variables Ü keras_api V Ż variables Žregularization_losses ßtrainable_variables ą keras_api „ Ykernel Y_kernel Zbias Z_bias į variables āregularization_losses ćtrainable_variables ä keras_api V å variables ęregularization_losses ētrainable_variables č keras_api v éaxis [gamma \beta ź variables ėregularization_losses ģtrainable_variables ķ keras_api v M0 N1 O2 P3 Q4 R5 S6 T7 U8 V9 W10 X11 Y12 Z13 [14 \15 v M0 N1 O2 P3 Q4 R5 S6 T7 U8 V9 W10 X11 Y12 Z13 [14 \15 µ ī layer_metrics “ variables ļnon_trainable_variables šlayers µregularization_losses ńmetrics ņlayer_regularization_losses ¶trainable_variables Ģ ó _query_dense ō _key_dense õ _value_dense ö_masked_softmax ÷_dropout_layer ¹ _output_dense ų variables łregularization_losses śtrainable_variables ū keras_api Ÿ üpartial_output_shape żfull_output_shape ckernel dbias ž variables ’regularization_losses €trainable_variables  keras_api V ‚ variables ƒregularization_losses „trainable_variables … keras_api v †axis egamma fbeta ‡ variables ˆregularization_losses ‰trainable_variables Š keras_api „ gkernel g_kernel hbias h_bias ‹ variables Œregularization_losses trainable_variables Ž keras_api V  variables regularization_losses ‘trainable_variables ’ keras_api „ ikernel i_kernel jbias j_bias “ variables ”regularization_losses •trainable_variables – keras_api V — variables ˜regularization_losses ™trainable_variables š keras_api v ›axis kgamma lbeta œ variables regularization_losses žtrainable_variables Ÿ keras_api v ]0 ^1 _2 `3 a4 b5 c6 d7 e8 f9 g10 h11 i12 j13 k14 l15 v ]0 ^1 _2 `3 a4 b5 c6 d7 e8 f9 g10 h11 i12 j13 k14 l15 µ   layer_metrics Į variables ”non_trainable_variables ¢layers Āregularization_losses £metrics ¤layer_regularization_losses Ćtrainable_variables Ģ „ _query_dense ¦ _key_dense § _value_dense Ø_masked_softmax ©_dropout_layer Ę _output_dense Ŗ variables «regularization_losses ¬trainable_variables ­ keras_api Ÿ ®partial_output_shape Æfull_output_shape skernel tbias ° variables ±regularization_losses ²trainable_variables ³ keras_api V “ variables µregularization_losses ¶trainable_variables · keras_api v øaxis ugamma vbeta ¹ variables ŗregularization_losses »trainable_variables ¼ keras_api „ wkernel w_kernel xbias x_bias ½ variables ¾regularization_losses ætrainable_variables Ą keras_api V Į variables Āregularization_losses Ćtrainable_variables Ä keras_api „ ykernel y_kernel zbias z_bias Å variables Ęregularization_losses Ētrainable_variables Č keras_api V É variables Źregularization_losses Ėtrainable_variables Ģ keras_api v Ķaxis {gamma |beta Ī variables Ļregularization_losses Štrainable_variables Ń keras_api v m0 n1 o2 p3 q4 r5 s6 t7 u8 v9 w10 x11 y12 z13 {14 |15 v m0 n1 o2 p3 q4 r5 s6 t7 u8 v9 w10 x11 y12 z13 {14 |15 µ Ņ layer_metrics Ī variables Ónon_trainable_variables Ōlayers Ļregularization_losses Õmetrics Ölayer_regularization_losses Štrainable_variables Ģ × _query_dense Ų _key_dense Ł _value_dense Ś_masked_softmax Ū_dropout_layer Ó _output_dense Ü variables Żregularization_losses Žtrainable_variables ß keras_api ” ąpartial_output_shape įfull_output_shape ƒkernel „bias ā variables ćregularization_losses ätrainable_variables å keras_api V ę variables ēregularization_losses čtrainable_variables é keras_api x źaxis …gamma †beta ė variables ģregularization_losses ķtrainable_variables ī keras_api ˆ ‡kernel ‡_kernel ˆbias ˆ_bias ļ variables šregularization_losses ńtrainable_variables ņ keras_api V ó variables ōregularization_losses õtrainable_variables ö keras_api ˆ ‰kernel ‰_kernel Šbias Š_bias ÷ variables ųregularization_losses łtrainable_variables ś keras_api V ū variables üregularization_losses żtrainable_variables ž keras_api x ’axis ‹gamma Œbeta € variables regularization_losses ‚trainable_variables ƒ keras_api ƒ }0 ~1 2 €3 4 ‚5 ƒ6 „7 …8 †9 ‡10 ˆ11 ‰12 Š13 ‹14 Œ15 ƒ }0 ~1 2 €3 4 ‚5 ƒ6 „7 …8 †9 ‡10 ˆ11 ‰12 Š13 ‹14 Œ15 µ „ layer_metrics Ū variables …non_trainable_variables †layers Üregularization_losses ‡metrics ˆlayer_regularization_losses Żtrainable_variables Ģ ‰ _query_dense Š _key_dense ‹ _value_dense Œ_masked_softmax _dropout_layer ą _output_dense Ž variables regularization_losses trainable_variables ‘ keras_api ” ’partial_output_shape “full_output_shape “kernel ”bias ” variables •regularization_losses –trainable_variables — keras_api V ˜ variables ™regularization_losses štrainable_variables › keras_api x œaxis •gamma –beta  variables žregularization_losses Ÿtrainable_variables   keras_api ˆ —kernel —_kernel ˜bias ˜_bias ” variables ¢regularization_losses £trainable_variables ¤ keras_api V „ variables ¦regularization_losses §trainable_variables Ø keras_api ˆ ™kernel ™_kernel šbias š_bias © variables Ŗregularization_losses «trainable_variables ¬ keras_api V ­ variables ®regularization_losses Ætrainable_variables ° keras_api x ±axis ›gamma œbeta ² variables ³regularization_losses “trainable_variables µ keras_api † 0 Ž1 2 3 ‘4 ’5 “6 ”7 •8 –9 —10 ˜11 ™12 š13 ›14 œ15 † 0 Ž1 2 3 ‘4 ’5 “6 ”7 •8 –9 —10 ˜11 ™12 š13 ›14 œ15 µ ¶ layer_metrics č variables ·non_trainable_variables ølayers éregularization_losses ¹metrics ŗlayer_regularization_losses źtrainable_variables Ģ » _query_dense ¼ _key_dense ½ _value_dense ¾_masked_softmax æ_dropout_layer ķ _output_dense Ą variables Įregularization_losses Ātrainable_variables Ć keras_api ” Äpartial_output_shape Åfull_output_shape £kernel ¤bias Ę variables Ēregularization_losses Čtrainable_variables É keras_api V Ź variables Ėregularization_losses Ģtrainable_variables Ķ keras_api x Īaxis „gamma ¦beta Ļ variables Šregularization_losses Ńtrainable_variables Ņ keras_api ˆ §kernel §_kernel Øbias Ø_bias Ó variables Ōregularization_losses Õtrainable_variables Ö keras_api V × variables Ųregularization_losses Łtrainable_variables Ś keras_api ˆ ©kernel ©_kernel Ŗbias Ŗ_bias Ū variables Üregularization_losses Żtrainable_variables Ž keras_api V ß variables ąregularization_losses įtrainable_variables ā keras_api x ćaxis «gamma ¬beta ä variables åregularization_losses ętrainable_variables ē keras_api † 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7 „8 ¦9 §10 Ø11 ©12 Ŗ13 «14 ¬15 † 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7 „8 ¦9 §10 Ø11 ©12 Ŗ13 «14 ¬15 µ č layer_metrics õ variables énon_trainable_variables źlayers öregularization_losses ėmetrics ģlayer_regularization_losses ÷trainable_variables Ģ ķ _query_dense ī _key_dense ļ _value_dense š_masked_softmax ń_dropout_layer ś _output_dense ņ variables óregularization_losses ōtrainable_variables õ keras_api ” öpartial_output_shape ÷full_output_shape ³kernel “bias ų variables łregularization_losses śtrainable_variables ū keras_api V ü variables żregularization_losses žtrainable_variables ’ keras_api x €axis µgamma ¶beta  variables ‚regularization_losses ƒtrainable_variables „ keras_api ˆ ·kernel ·_kernel øbias ø_bias … variables †regularization_losses ‡trainable_variables ˆ keras_api V ‰ variables Šregularization_losses ‹trainable_variables Œ keras_api ˆ ¹kernel ¹_kernel ŗbias ŗ_bias  variables Žregularization_losses trainable_variables  keras_api V ‘ variables ’regularization_losses “trainable_variables ” keras_api x •axis »gamma ¼beta – variables —regularization_losses ˜trainable_variables ™ keras_api † ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7 µ8 ¶9 ·10 ø11 ¹12 ŗ13 »14 ¼15 † ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7 µ8 ¶9 ·10 ø11 ¹12 ŗ13 »14 ¼15 µ š layer_metrics ‚ variables ›non_trainable_variables œlayers ƒregularization_losses metrics žlayer_regularization_losses „trainable_variables Ģ Ÿ _query_dense   _key_dense ” _value_dense ¢_masked_softmax £_dropout_layer ‡ _output_dense ¤ variables „regularization_losses ¦trainable_variables § keras_api ” Øpartial_output_shape ©full_output_shape Ćkernel Äbias Ŗ variables «regularization_losses ¬trainable_variables ­ keras_api V ® variables Æregularization_losses °trainable_variables ± keras_api x ²axis Ågamma Ębeta ³ variables “regularization_losses µtrainable_variables ¶ keras_api ˆ Ēkernel Ē_kernel Čbias Č_bias · variables øregularization_losses ¹trainable_variables ŗ keras_api V » variables ¼regularization_losses ½trainable_variables ¾ keras_api ˆ Ékernel É_kernel Źbias Ź_bias æ variables Ąregularization_losses Įtrainable_variables Ā keras_api V Ć variables Äregularization_losses Åtrainable_variables Ę keras_api x Ēaxis Ėgamma Ģbeta Č variables Éregularization_losses Źtrainable_variables Ė keras_api † ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7 Å8 Ę9 Ē10 Č11 É12 Ź13 Ė14 Ģ15 † ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7 Å8 Ę9 Ē10 Č11 É12 Ź13 Ė14 Ģ15 µ Ģ layer_metrics  variables Ķnon_trainable_variables Īlayers regularization_losses Ļmetrics Šlayer_regularization_losses ‘trainable_variables Ģ Ń _query_dense Ņ _key_dense Ó _value_dense Ō_masked_softmax Õ_dropout_layer ” _output_dense Ö variables ×regularization_losses Ųtrainable_variables Ł keras_api ” Śpartial_output_shape Ūfull_output_shape Ókernel Ōbias Ü variables Żregularization_losses Žtrainable_variables ß keras_api V ą variables įregularization_losses ātrainable_variables ć keras_api x äaxis Õgamma Öbeta å variables ęregularization_losses ētrainable_variables č keras_api ˆ ×kernel ×_kernel Ųbias Ų_bias é variables źregularization_losses ėtrainable_variables ģ keras_api V ķ variables īregularization_losses ļtrainable_variables š keras_api ˆ Łkernel Ł_kernel Śbias Ś_bias ń variables ņregularization_losses ótrainable_variables ō keras_api V õ variables öregularization_losses ÷trainable_variables ų keras_api x łaxis Ūgamma Übeta ś variables ūregularization_losses ütrainable_variables ż keras_api † Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7 Õ8 Ö9 ×10 Ų11 Ł12 Ś13 Ū14 Ü15 † Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7 Õ8 Ö9 ×10 Ų11 Ł12 Ś13 Ū14 Ü15 µ ž layer_metrics œ variables ’non_trainable_variables €layers regularization_losses metrics ‚layer_regularization_losses žtrainable_variables Ģ ƒ _query_dense „ _key_dense … _value_dense †_masked_softmax ‡_dropout_layer ” _output_dense ˆ variables ‰regularization_losses Štrainable_variables ‹ keras_api ” Œpartial_output_shape full_output_shape ćkernel äbias Ž variables regularization_losses trainable_variables ‘ keras_api V ’ variables “regularization_losses ”trainable_variables • keras_api x –axis ågamma ębeta — variables ˜regularization_losses ™trainable_variables š keras_api ˆ ēkernel ē_kernel čbias č_bias › variables œregularization_losses trainable_variables ž keras_api V Ÿ variables  regularization_losses ”trainable_variables ¢ keras_api ˆ ékernel é_kernel źbias ź_bias £ variables ¤regularization_losses „trainable_variables ¦ keras_api V § variables Øregularization_losses ©trainable_variables Ŗ keras_api x «axis ėgamma ģbeta ¬ variables ­regularization_losses ®trainable_variables Æ keras_api † Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7 å8 ę9 ē10 č11 é12 ź13 ė14 ģ15 † Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7 å8 ę9 ē10 č11 é12 ź13 ė14 ģ15 µ ° layer_metrics © variables ±non_trainable_variables ²layers Ŗregularization_losses ³metrics “layer_regularization_losses «trainable_variables µ µ layer_metrics ­ variables ¶non_trainable_variables ·layers ®regularization_losses ømetrics ¹layer_regularization_losses Ætrainable_variables  ķ0 ī1  ķ0 ī1 µ ŗ layer_metrics ± variables »non_trainable_variables ¼layers ²regularization_losses ½metrics ¾layer_regularization_losses ³trainable_variables ¶  0  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19  20 !21 "22 #23 Ÿ æpartial_output_shape Ąfull_output_shape -kernel .bias Į variables Āregularization_losses Ćtrainable_variables Ä keras_api Ÿ Åpartial_output_shape Ęfull_output_shape /kernel 0bias Ē variables Čregularization_losses Étrainable_variables Ź keras_api Ÿ Ėpartial_output_shape Ģfull_output_shape 1kernel 2bias Ķ variables Īregularization_losses Ļtrainable_variables Š keras_api q Ń_mask_expansion_axes Ņ variables Óregularization_losses Ōtrainable_variables Õ keras_api V Ö variables ×regularization_losses Ųtrainable_variables Ł keras_api 8 -0 .1 /2 03 14 25 36 47 8 -0 .1 /2 03 14 25 36 47 µ Ś layer_metrics ā variables Ūnon_trainable_variables Ülayers ćregularization_losses Żmetrics Žlayer_regularization_losses ätrainable_variables  30 41  30 41 µ ß layer_metrics č variables ąnon_trainable_variables įlayers éregularization_losses āmetrics ćlayer_regularization_losses źtrainable_variables µ ä layer_metrics ģ variables ånon_trainable_variables ęlayers ķregularization_losses ēmetrics člayer_regularization_losses ītrainable_variables  50 61  50 61 µ é layer_metrics ń variables źnon_trainable_variables ėlayers ņregularization_losses ģmetrics ķlayer_regularization_losses ótrainable_variables  70 81  70 81 µ ī layer_metrics õ variables ļnon_trainable_variables šlayers öregularization_losses ńmetrics ņlayer_regularization_losses ÷trainable_variables µ ó layer_metrics ł variables ōnon_trainable_variables õlayers śregularization_losses ömetrics ÷layer_regularization_losses ūtrainable_variables  90 :1  90 :1 µ ų layer_metrics ż variables łnon_trainable_variables ślayers žregularization_losses ūmetrics ülayer_regularization_losses ’trainable_variables µ ż layer_metrics  variables žnon_trainable_variables ’layers ‚regularization_losses € metrics  layer_regularization_losses ƒtrainable_variables  ;0 <1  ;0 <1 µ ‚  layer_metrics † variables ƒ non_trainable_variables „ layers ‡regularization_losses … metrics † layer_regularization_losses ˆtrainable_variables H ‘0 ’1 “2 ”3 •4 –5 —6 ˜7 ™8 Ÿ ‡ partial_output_shape ˆ full_output_shape =kernel >bias ‰  variables Š regularization_losses ‹ trainable_variables Œ  keras_api Ÿ  partial_output_shape Ž full_output_shape ?kernel @bias   variables  regularization_losses ‘ trainable_variables ’  keras_api Ÿ “ partial_output_shape ” full_output_shape Akernel Bbias •  variables – regularization_losses — trainable_variables ˜  keras_api q ™ _mask_expansion_axes š  variables › regularization_losses œ trainable_variables   keras_api V ž  variables Ÿ regularization_losses   trainable_variables ”  keras_api 8 =0 >1 ?2 @3 A4 B5 C6 D7 8 =0 >1 ?2 @3 A4 B5 C6 D7 µ ¢  layer_metrics ” variables £ non_trainable_variables ¤ layers •regularization_losses „ metrics ¦ layer_regularization_losses –trainable_variables  C0 D1  C0 D1 µ §  layer_metrics š variables Ø non_trainable_variables © layers ›regularization_losses Ŗ metrics « layer_regularization_losses œtrainable_variables µ ¬  layer_metrics ž variables ­ non_trainable_variables ® layers Ÿregularization_losses Æ metrics ° layer_regularization_losses  trainable_variables  E0 F1  E0 F1 µ ±  layer_metrics £ variables ² non_trainable_variables ³ layers ¤regularization_losses “ metrics µ layer_regularization_losses „trainable_variables  G0 H1  G0 H1 µ ¶  layer_metrics § variables · non_trainable_variables ø layers Øregularization_losses ¹ metrics ŗ layer_regularization_losses ©trainable_variables µ »  layer_metrics « variables ¼ non_trainable_variables ½ layers ¬regularization_losses ¾ metrics æ layer_regularization_losses ­trainable_variables  I0 J1  I0 J1 µ Ą  layer_metrics Æ variables Į non_trainable_variables Ā layers °regularization_losses Ć metrics Ä layer_regularization_losses ±trainable_variables µ Å  layer_metrics ³ variables Ę non_trainable_variables Ē layers “regularization_losses Č metrics É layer_regularization_losses µtrainable_variables  K0 L1  K0 L1 µ Ź  layer_metrics ø variables Ė non_trainable_variables Ģ layers ¹regularization_losses Ķ metrics Ī layer_regularization_losses ŗtrainable_variables H ž0 Ÿ1  2 ”3 ¢4 £5 ¤6 „7 ¦8 Ÿ Ļ partial_output_shape Š full_output_shape Mkernel Nbias Ń  variables Ņ regularization_losses Ó trainable_variables Ō  keras_api Ÿ Õ partial_output_shape Ö full_output_shape Okernel Pbias ×  variables Ų regularization_losses Ł trainable_variables Ś  keras_api Ÿ Ū partial_output_shape Ü full_output_shape Qkernel Rbias Ż  variables Ž regularization_losses ß trainable_variables ą  keras_api q į _mask_expansion_axes ā  variables ć regularization_losses ä trainable_variables å  keras_api V ę  variables ē regularization_losses č trainable_variables é  keras_api 8 M0 N1 O2 P3 Q4 R5 S6 T7 8 M0 N1 O2 P3 Q4 R5 S6 T7 µ ź  layer_metrics Ę variables ė non_trainable_variables ģ layers Ēregularization_losses ķ metrics ī layer_regularization_losses Čtrainable_variables  S0 T1  S0 T1 µ ļ  layer_metrics Ģ variables š non_trainable_variables ń layers Ķregularization_losses ņ metrics ó layer_regularization_losses Ītrainable_variables µ ō  layer_metrics Š variables õ non_trainable_variables ö layers Ńregularization_losses ÷ metrics ų layer_regularization_losses Ņtrainable_variables  U0 V1  U0 V1 µ ł  layer_metrics Õ variables ś non_trainable_variables ū layers Öregularization_losses ü metrics ż layer_regularization_losses ×trainable_variables  W0 X1  W0 X1 µ ž  layer_metrics Ł variables ’ non_trainable_variables € layers Śregularization_losses  metrics ‚ layer_regularization_losses Ūtrainable_variables µ ƒ  layer_metrics Ż variables „ non_trainable_variables … layers Žregularization_losses † metrics ‡ layer_regularization_losses ßtrainable_variables  Y0 Z1  Y0 Z1 µ ˆ  layer_metrics į variables ‰ non_trainable_variables Š layers āregularization_losses ‹ metrics Œ layer_regularization_losses ćtrainable_variables µ   layer_metrics å variables Ž non_trainable_variables  layers ęregularization_losses  metrics ‘ layer_regularization_losses ētrainable_variables  [0 \1  [0 \1 µ ’  layer_metrics ź variables “ non_trainable_variables ” layers ėregularization_losses • metrics – layer_regularization_losses ģtrainable_variables H «0 ¬1 ­2 ®3 Æ4 °5 ±6 ²7 ³8 Ÿ — partial_output_shape ˜ full_output_shape ]kernel ^bias ™  variables š regularization_losses › trainable_variables œ  keras_api Ÿ  partial_output_shape ž full_output_shape _kernel `bias Ÿ  variables   regularization_losses ” trainable_variables ¢  keras_api Ÿ £ partial_output_shape ¤ full_output_shape akernel bbias „  variables ¦ regularization_losses § trainable_variables Ø  keras_api q © _mask_expansion_axes Ŗ  variables « regularization_losses ¬ trainable_variables ­  keras_api V ®  variables Æ regularization_losses ° trainable_variables ±  keras_api 8 ]0 ^1 _2 `3 a4 b5 c6 d7 8 ]0 ^1 _2 `3 a4 b5 c6 d7 µ ²  layer_metrics ų variables ³ non_trainable_variables “ layers łregularization_losses µ metrics ¶ layer_regularization_losses śtrainable_variables  c0 d1  c0 d1 µ ·  layer_metrics ž variables ø non_trainable_variables ¹ layers ’regularization_losses ŗ metrics » layer_regularization_losses €trainable_variables µ ¼  layer_metrics ‚ variables ½ non_trainable_variables ¾ layers ƒregularization_losses æ metrics Ą layer_regularization_losses „trainable_variables  e0 f1  e0 f1 µ Į  layer_metrics ‡ variables Ā non_trainable_variables Ć layers ˆregularization_losses Ä metrics Å layer_regularization_losses ‰trainable_variables  g0 h1  g0 h1 µ Ę  layer_metrics ‹ variables Ē non_trainable_variables Č layers Œregularization_losses É metrics Ź layer_regularization_losses trainable_variables µ Ė  layer_metrics  variables Ģ non_trainable_variables Ķ layers regularization_losses Ī metrics Ļ layer_regularization_losses ‘trainable_variables  i0 j1  i0 j1 µ Š  layer_metrics “ variables Ń non_trainable_variables Ņ layers ”regularization_losses Ó metrics Ō layer_regularization_losses •trainable_variables µ Õ  layer_metrics — variables Ö non_trainable_variables × layers ˜regularization_losses Ų metrics Ł layer_regularization_losses ™trainable_variables  k0 l1  k0 l1 µ Ś  layer_metrics œ variables Ū non_trainable_variables Ü layers regularization_losses Ż metrics Ž layer_regularization_losses žtrainable_variables H ø0 ¹1 ŗ2 »3 ¼4 ½5 ¾6 æ7 Ą8 Ÿ ß partial_output_shape ą full_output_shape mkernel nbias į  variables ā regularization_losses ć trainable_variables ä  keras_api Ÿ å partial_output_shape ę full_output_shape okernel pbias ē  variables č regularization_losses é trainable_variables ź  keras_api Ÿ ė partial_output_shape ģ full_output_shape qkernel rbias ķ  variables ī regularization_losses ļ trainable_variables š  keras_api q ń _mask_expansion_axes ņ  variables ó regularization_losses ō trainable_variables õ  keras_api V ö  variables ÷ regularization_losses ų trainable_variables ł  keras_api 8 m0 n1 o2 p3 q4 r5 s6 t7 8 m0 n1 o2 p3 q4 r5 s6 t7 µ ś  layer_metrics Ŗ variables ū non_trainable_variables ü layers «regularization_losses ż metrics ž layer_regularization_losses ¬trainable_variables  s0 t1  s0 t1 µ ’  layer_metrics ° variables € non_trainable_variables  layers ±regularization_losses ‚ metrics ƒ layer_regularization_losses ²trainable_variables µ „  layer_metrics “ variables … non_trainable_variables † layers µregularization_losses ‡ metrics ˆ layer_regularization_losses ¶trainable_variables  u0 v1  u0 v1 µ ‰  layer_metrics ¹ variables Š non_trainable_variables ‹ layers ŗregularization_losses Œ metrics  layer_regularization_losses »trainable_variables  w0 x1  w0 x1 µ Ž  layer_metrics ½ variables  non_trainable_variables  layers ¾regularization_losses ‘ metrics ’ layer_regularization_losses ætrainable_variables µ “  layer_metrics Į variables ” non_trainable_variables • layers Āregularization_losses – metrics — layer_regularization_losses Ćtrainable_variables  y0 z1  y0 z1 µ ˜  layer_metrics Å variables ™ non_trainable_variables š layers Ęregularization_losses › metrics œ layer_regularization_losses Ētrainable_variables µ   layer_metrics É variables ž non_trainable_variables Ÿ layers Źregularization_losses   metrics ” layer_regularization_losses Ėtrainable_variables  {0 |1  {0 |1 µ ¢  layer_metrics Ī variables £ non_trainable_variables ¤ layers Ļregularization_losses „ metrics ¦ layer_regularization_losses Štrainable_variables H Å0 Ę1 Ē2 Č3 É4 Ź5 Ė6 Ģ7 Ķ8 Ÿ § partial_output_shape Ø full_output_shape }kernel ~bias ©  variables Ŗ regularization_losses « trainable_variables ¬  keras_api   ­ partial_output_shape ® full_output_shape kernel €bias Æ  variables ° regularization_losses ± trainable_variables ²  keras_api ” ³ partial_output_shape “ full_output_shape kernel ‚bias µ  variables ¶ regularization_losses · trainable_variables ø  keras_api q ¹ _mask_expansion_axes ŗ  variables » regularization_losses ¼ trainable_variables ½  keras_api V ¾  variables æ regularization_losses Ą trainable_variables Į  keras_api = }0 ~1 2 €3 4 ‚5 ƒ6 „7 = }0 ~1 2 €3 4 ‚5 ƒ6 „7 µ Ā  layer_metrics Ü variables Ć non_trainable_variables Ä layers Żregularization_losses Å metrics Ę layer_regularization_losses Žtrainable_variables  ƒ0 „1  ƒ0 „1 µ Ē  layer_metrics ā variables Č non_trainable_variables É layers ćregularization_losses Ź metrics Ė layer_regularization_losses ätrainable_variables µ Ģ  layer_metrics ę variables Ķ non_trainable_variables Ī layers ēregularization_losses Ļ metrics Š layer_regularization_losses čtrainable_variables  …0 †1  …0 †1 µ Ń  layer_metrics ė variables Ņ non_trainable_variables Ó layers ģregularization_losses Ō metrics Õ layer_regularization_losses ķtrainable_variables  ‡0 ˆ1  ‡0 ˆ1 µ Ö  layer_metrics ļ variables × non_trainable_variables Ų layers šregularization_losses Ł metrics Ś layer_regularization_losses ńtrainable_variables µ Ū  layer_metrics ó variables Ü non_trainable_variables Ż layers ōregularization_losses Ž metrics ß layer_regularization_losses õtrainable_variables  ‰0 Š1  ‰0 Š1 µ ą  layer_metrics ÷ variables į non_trainable_variables ā layers ųregularization_losses ć metrics ä layer_regularization_losses łtrainable_variables µ å  layer_metrics ū variables ę non_trainable_variables ē layers üregularization_losses č metrics é layer_regularization_losses żtrainable_variables  ‹0 Œ1  ‹0 Œ1 µ ź  layer_metrics € variables ė non_trainable_variables ģ layers regularization_losses ķ metrics ī layer_regularization_losses ‚trainable_variables H Ņ0 Ó1 Ō2 Õ3 Ö4 ×5 Ų6 Ł7 Ś8 ” ļ partial_output_shape š full_output_shape kernel Žbias ń  variables ņ regularization_losses ó trainable_variables ō  keras_api ” õ partial_output_shape ö full_output_shape kernel bias ÷  variables ų regularization_losses ł trainable_variables ś  keras_api ” ū partial_output_shape ü full_output_shape ‘kernel ’bias ż  variables ž regularization_losses ’ trainable_variables €  keras_api q  _mask_expansion_axes ‚  variables ƒ regularization_losses „ trainable_variables …  keras_api V †  variables ‡ regularization_losses ˆ trainable_variables ‰  keras_api @ 0 Ž1 2 3 ‘4 ’5 “6 ”7 @ 0 Ž1 2 3 ‘4 ’5 “6 ”7 µ Š  layer_metrics Ž variables ‹ non_trainable_variables Œ layers regularization_losses  metrics Ž layer_regularization_losses trainable_variables  “0 ”1  “0 ”1 µ   layer_metrics ” variables  non_trainable_variables ‘ layers •regularization_losses ’ metrics “ layer_regularization_losses –trainable_variables µ ”  layer_metrics ˜ variables • non_trainable_variables – layers ™regularization_losses — metrics ˜ layer_regularization_losses štrainable_variables  •0 –1  •0 –1 µ ™  layer_metrics  variables š non_trainable_variables › layers žregularization_losses œ metrics  layer_regularization_losses Ÿtrainable_variables  —0 ˜1  —0 ˜1 µ ž  layer_metrics ” variables Ÿ non_trainable_variables   layers ¢regularization_losses ” metrics ¢ layer_regularization_losses £trainable_variables µ £  layer_metrics „ variables ¤ non_trainable_variables „ layers ¦regularization_losses ¦ metrics § layer_regularization_losses §trainable_variables  ™0 š1  ™0 š1 µ Ø  layer_metrics © variables © non_trainable_variables Ŗ layers Ŗregularization_losses « metrics ¬ layer_regularization_losses «trainable_variables µ ­  layer_metrics ­ variables ® non_trainable_variables Æ layers ®regularization_losses ° metrics ± layer_regularization_losses Ætrainable_variables  ›0 œ1  ›0 œ1 µ ²  layer_metrics ² variables ³ non_trainable_variables “ layers ³regularization_losses µ metrics ¶ layer_regularization_losses “trainable_variables H ß0 ą1 į2 ā3 ć4 ä5 å6 ę7 ē8 ” · partial_output_shape ø full_output_shape kernel žbias ¹  variables ŗ regularization_losses » trainable_variables ¼  keras_api ” ½ partial_output_shape ¾ full_output_shape Ÿkernel  bias æ  variables Ą regularization_losses Į trainable_variables Ā  keras_api ” Ć partial_output_shape Ä full_output_shape ”kernel ¢bias Å  variables Ę regularization_losses Ē trainable_variables Č  keras_api q É _mask_expansion_axes Ź  variables Ė regularization_losses Ģ trainable_variables Ķ  keras_api V Ī  variables Ļ regularization_losses Š trainable_variables Ń  keras_api @ 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7 @ 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7 µ Ņ  layer_metrics Ą variables Ó non_trainable_variables Ō layers Įregularization_losses Õ metrics Ö layer_regularization_losses Ātrainable_variables  £0 ¤1  £0 ¤1 µ ×  layer_metrics Ę variables Ų non_trainable_variables Ł layers Ēregularization_losses Ś metrics Ū layer_regularization_losses Čtrainable_variables µ Ü  layer_metrics Ź variables Ż non_trainable_variables Ž layers Ėregularization_losses ß metrics ą layer_regularization_losses Ģtrainable_variables  „0 ¦1  „0 ¦1 µ į  layer_metrics Ļ variables ā non_trainable_variables ć layers Šregularization_losses ä metrics å layer_regularization_losses Ńtrainable_variables  §0 Ø1  §0 Ø1 µ ę  layer_metrics Ó variables ē non_trainable_variables č layers Ōregularization_losses é metrics ź layer_regularization_losses Õtrainable_variables µ ė  layer_metrics × variables ģ non_trainable_variables ķ layers Ųregularization_losses ī metrics ļ layer_regularization_losses Łtrainable_variables  ©0 Ŗ1  ©0 Ŗ1 µ š  layer_metrics Ū variables ń non_trainable_variables ņ layers Üregularization_losses ó metrics ō layer_regularization_losses Żtrainable_variables µ õ  layer_metrics ß variables ö non_trainable_variables ÷ layers ąregularization_losses ų metrics ł layer_regularization_losses įtrainable_variables  «0 ¬1  «0 ¬1 µ ś  layer_metrics ä variables ū non_trainable_variables ü layers åregularization_losses ż metrics ž layer_regularization_losses ętrainable_variables H ģ0 ķ1 ī2 ļ3 š4 ń5 ņ6 ó7 ō8 ” ’ partial_output_shape € full_output_shape ­kernel ®bias   variables ‚ regularization_losses ƒ trainable_variables „  keras_api ” … partial_output_shape † full_output_shape Ækernel °bias ‡  variables ˆ regularization_losses ‰ trainable_variables Š  keras_api ” ‹ partial_output_shape Œ full_output_shape ±kernel ²bias   variables Ž regularization_losses  trainable_variables   keras_api q ‘ _mask_expansion_axes ’  variables “ regularization_losses ” trainable_variables •  keras_api V –  variables — regularization_losses ˜ trainable_variables ™  keras_api @ ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7 @ ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7 µ š  layer_metrics ņ variables › non_trainable_variables œ layers óregularization_losses  metrics ž layer_regularization_losses ōtrainable_variables  ³0 “1  ³0 “1 µ Ÿ  layer_metrics ų variables   non_trainable_variables ” layers łregularization_losses ¢ metrics £ layer_regularization_losses śtrainable_variables µ ¤  layer_metrics ü variables „ non_trainable_variables ¦ layers żregularization_losses § metrics Ø layer_regularization_losses žtrainable_variables  µ0 ¶1  µ0 ¶1 µ ©  layer_metrics  variables Ŗ non_trainable_variables « layers ‚regularization_losses ¬ metrics ­ layer_regularization_losses ƒtrainable_variables  ·0 ø1  ·0 ø1 µ ®  layer_metrics … variables Æ non_trainable_variables ° layers †regularization_losses ± metrics ² layer_regularization_losses ‡trainable_variables µ ³  layer_metrics ‰ variables “ non_trainable_variables µ layers Šregularization_losses ¶ metrics · layer_regularization_losses ‹trainable_variables  ¹0 ŗ1  ¹0 ŗ1 µ ø  layer_metrics  variables ¹ non_trainable_variables ŗ layers Žregularization_losses » metrics ¼ layer_regularization_losses trainable_variables µ ½  layer_metrics ‘ variables ¾ non_trainable_variables æ layers ’regularization_losses Ą metrics Į layer_regularization_losses “trainable_variables  »0 ¼1  »0 ¼1 µ Ā  layer_metrics – variables Ć non_trainable_variables Ä layers —regularization_losses Å metrics Ę layer_regularization_losses ˜trainable_variables H ł0 ś1 ū2 ü3 ż4 ž5 ’6 €7 8 ” Ē partial_output_shape Č full_output_shape ½kernel ¾bias É  variables Ź regularization_losses Ė trainable_variables Ģ  keras_api ” Ķ partial_output_shape Ī full_output_shape ækernel Ąbias Ļ  variables Š regularization_losses Ń trainable_variables Ņ  keras_api ” Ó partial_output_shape Ō full_output_shape Įkernel Ābias Õ  variables Ö regularization_losses × trainable_variables Ų  keras_api q Ł _mask_expansion_axes Ś  variables Ū regularization_losses Ü trainable_variables Ż  keras_api V Ž  variables ß regularization_losses ą trainable_variables į  keras_api @ ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7 @ ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7 µ ā  layer_metrics ¤ variables ć non_trainable_variables ä layers „regularization_losses å metrics ę layer_regularization_losses ¦trainable_variables  Ć0 Ä1  Ć0 Ä1 µ ē  layer_metrics Ŗ variables č non_trainable_variables é layers «regularization_losses ź metrics ė layer_regularization_losses ¬trainable_variables µ ģ  layer_metrics ® variables ķ non_trainable_variables ī layers Æregularization_losses ļ metrics š layer_regularization_losses °trainable_variables  Å0 Ę1  Å0 Ę1 µ ń  layer_metrics ³ variables ņ non_trainable_variables ó layers “regularization_losses ō metrics õ layer_regularization_losses µtrainable_variables  Ē0 Č1  Ē0 Č1 µ ö  layer_metrics · variables ÷ non_trainable_variables ų layers øregularization_losses ł metrics ś layer_regularization_losses ¹trainable_variables µ ū  layer_metrics » variables ü non_trainable_variables ż layers ¼regularization_losses ž metrics ’ layer_regularization_losses ½trainable_variables  É0 Ź1  É0 Ź1 µ € layer_metrics æ variables non_trainable_variables ‚layers Ąregularization_losses ƒmetrics „layer_regularization_losses Įtrainable_variables µ … layer_metrics Ć variables †non_trainable_variables ‡layers Äregularization_losses ˆmetrics ‰layer_regularization_losses Åtrainable_variables  Ė0 Ģ1  Ė0 Ģ1 µ Š layer_metrics Č variables ‹non_trainable_variables Œlayers Éregularization_losses metrics Žlayer_regularization_losses Źtrainable_variables H †0 ‡1 ˆ2 ‰3 Š4 ‹5 Œ6 7 Ž8 ” partial_output_shape full_output_shape Ķkernel Ībias ‘ variables ’regularization_losses “trainable_variables ” keras_api ” •partial_output_shape –full_output_shape Ļkernel Šbias — variables ˜regularization_losses ™trainable_variables š keras_api ” ›partial_output_shape œfull_output_shape Ńkernel Ņbias  variables žregularization_losses Ÿtrainable_variables   keras_api q ”_mask_expansion_axes ¢ variables £regularization_losses ¤trainable_variables „ keras_api V ¦ variables §regularization_losses Øtrainable_variables © keras_api @ Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7 @ Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7 µ Ŗ layer_metrics Ö variables «non_trainable_variables ¬layers ×regularization_losses ­metrics ®layer_regularization_losses Ųtrainable_variables  Ó0 Ō1  Ó0 Ō1 µ Æ layer_metrics Ü variables °non_trainable_variables ±layers Żregularization_losses ²metrics ³layer_regularization_losses Žtrainable_variables µ “ layer_metrics ą variables µnon_trainable_variables ¶layers įregularization_losses ·metrics ølayer_regularization_losses ātrainable_variables  Õ0 Ö1  Õ0 Ö1 µ ¹ layer_metrics å variables ŗnon_trainable_variables »layers ęregularization_losses ¼metrics ½layer_regularization_losses ētrainable_variables  ×0 Ų1  ×0 Ų1 µ ¾ layer_metrics é variables ænon_trainable_variables Ąlayers źregularization_losses Įmetrics Ālayer_regularization_losses ėtrainable_variables µ Ć layer_metrics ķ variables Änon_trainable_variables Ålayers īregularization_losses Ęmetrics Ēlayer_regularization_losses ļtrainable_variables  Ł0 Ś1  Ł0 Ś1 µ Č layer_metrics ń variables Énon_trainable_variables Źlayers ņregularization_losses Ėmetrics Ģlayer_regularization_losses ótrainable_variables µ Ķ layer_metrics õ variables Īnon_trainable_variables Ļlayers öregularization_losses Šmetrics Ńlayer_regularization_losses ÷trainable_variables  Ū0 Ü1  Ū0 Ü1 µ Ņ layer_metrics ś variables Ónon_trainable_variables Ōlayers ūregularization_losses Õmetrics Ölayer_regularization_losses ütrainable_variables H “0 ”1 •2 –3 —4 ˜5 ™6 š7 ›8 ” ×partial_output_shape Ųfull_output_shape Żkernel Žbias Ł variables Śregularization_losses Ūtrainable_variables Ü keras_api ” Żpartial_output_shape Žfull_output_shape ßkernel ąbias ß variables ąregularization_losses įtrainable_variables ā keras_api ” ćpartial_output_shape äfull_output_shape įkernel ābias å variables ęregularization_losses ētrainable_variables č keras_api q é_mask_expansion_axes ź variables ėregularization_losses ģtrainable_variables ķ keras_api V ī variables ļregularization_losses štrainable_variables ń keras_api @ Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7 @ Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7 µ ņ layer_metrics ˆ variables ónon_trainable_variables ōlayers ‰regularization_losses õmetrics ölayer_regularization_losses Štrainable_variables  ć0 ä1  ć0 ä1 µ ÷ layer_metrics Ž variables ųnon_trainable_variables łlayers regularization_losses śmetrics ūlayer_regularization_losses trainable_variables µ ü layer_metrics ’ variables żnon_trainable_variables žlayers “regularization_losses ’metrics €layer_regularization_losses ”trainable_variables  å0 ę1  å0 ę1 µ  layer_metrics — variables ‚non_trainable_variables ƒlayers ˜regularization_losses „metrics …layer_regularization_losses ™trainable_variables  ē0 č1  ē0 č1 µ † layer_metrics › variables ‡non_trainable_variables ˆlayers œregularization_losses ‰metrics Šlayer_regularization_losses trainable_variables µ ‹ layer_metrics Ÿ variables Œnon_trainable_variables layers  regularization_losses Žmetrics layer_regularization_losses ”trainable_variables  é0 ź1  é0 ź1 µ  layer_metrics £ variables ‘non_trainable_variables ’layers ¤regularization_losses “metrics ”layer_regularization_losses „trainable_variables µ • layer_metrics § variables –non_trainable_variables —layers Øregularization_losses ˜metrics ™layer_regularization_losses ©trainable_variables  ė0 ģ1  ė0 ģ1 µ š layer_metrics ¬ variables ›non_trainable_variables œlayers ­regularization_losses metrics žlayer_regularization_losses ®trainable_variables H  0 ”1 ¢2 £3 ¤4 „5 ¦6 §7 Ø8  -0 .1  -0 .1 µ Ÿ layer_metrics Į variables  non_trainable_variables ”layers Āregularization_losses ¢metrics £layer_regularization_losses Ćtrainable_variables  /0 01  /0 01 µ ¤ layer_metrics Ē variables „non_trainable_variables ¦layers Čregularization_losses §metrics Ølayer_regularization_losses Étrainable_variables  10 21  10 21 µ © layer_metrics Ķ variables Ŗnon_trainable_variables «layers Īregularization_losses ¬metrics ­layer_regularization_losses Ļtrainable_variables µ ® layer_metrics Ņ variables Ænon_trainable_variables °layers Óregularization_losses ±metrics ²layer_regularization_losses Ōtrainable_variables µ ³ layer_metrics Ö variables “non_trainable_variables µlayers ×regularization_losses ¶metrics ·layer_regularization_losses Ųtrainable_variables 0 Ż0 Ž1 ß2 ą3 į4 ’5  =0 >1  =0 >1 µ ø layer_metrics ‰  variables ¹non_trainable_variables ŗlayers Š regularization_losses »metrics ¼layer_regularization_losses ‹ trainable_variables  ?0 @1  ?0 @1 µ ½ layer_metrics   variables ¾non_trainable_variables ælayers  regularization_losses Ąmetrics Įlayer_regularization_losses ‘ trainable_variables  A0 B1  A0 B1 µ Ā layer_metrics •  variables Ćnon_trainable_variables Älayers – regularization_losses Åmetrics Ęlayer_regularization_losses — trainable_variables µ Ē layer_metrics š  variables Čnon_trainable_variables Élayers › regularization_losses Źmetrics Ėlayer_regularization_losses œ trainable_variables µ Ģ layer_metrics ž  variables Ķnon_trainable_variables Īlayers Ÿ regularization_losses Ļmetrics Šlayer_regularization_losses   trainable_variables 0 0 1 ‘2 ’3 “4 Ÿ5  M0 N1  M0 N1 µ Ń layer_metrics Ń  variables Ņnon_trainable_variables Ólayers Ņ regularization_losses Ōmetrics Õlayer_regularization_losses Ó trainable_variables  O0 P1  O0 P1 µ Ö layer_metrics ×  variables ×non_trainable_variables Ųlayers Ų regularization_losses Łmetrics Ślayer_regularization_losses Ł trainable_variables  Q0 R1  Q0 R1 µ Ū layer_metrics Ż  variables Ünon_trainable_variables Żlayers Ž regularization_losses Žmetrics ßlayer_regularization_losses ß trainable_variables µ ą layer_metrics ā  variables įnon_trainable_variables ālayers ć regularization_losses ćmetrics älayer_regularization_losses ä trainable_variables µ å layer_metrics ę  variables ęnon_trainable_variables ēlayers ē regularization_losses čmetrics élayer_regularization_losses č trainable_variables 0 Į0 Ā1 Ć2 Ä3 Å4 ¬5  ]0 ^1  ]0 ^1 µ ź layer_metrics ™  variables ėnon_trainable_variables ģlayers š regularization_losses ķmetrics īlayer_regularization_losses › trainable_variables  _0 `1  _0 `1 µ ļ layer_metrics Ÿ  variables šnon_trainable_variables ńlayers   regularization_losses ņmetrics ólayer_regularization_losses ” trainable_variables  a0 b1  a0 b1 µ ō layer_metrics „  variables õnon_trainable_variables ölayers ¦ regularization_losses ÷metrics ųlayer_regularization_losses § trainable_variables µ ł layer_metrics Ŗ  variables śnon_trainable_variables ūlayers « regularization_losses ümetrics żlayer_regularization_losses ¬ trainable_variables µ ž layer_metrics ®  variables ’non_trainable_variables €layers Æ regularization_losses metrics ‚layer_regularization_losses ° trainable_variables 0 ó0 ō1 õ2 ö3 ÷4 ¹5  m0 n1  m0 n1 µ ƒ layer_metrics į  variables „non_trainable_variables …layers ā regularization_losses †metrics ‡layer_regularization_losses ć trainable_variables  o0 p1  o0 p1 µ ˆ layer_metrics ē  variables ‰non_trainable_variables Šlayers č regularization_losses ‹metrics Œlayer_regularization_losses é trainable_variables  q0 r1  q0 r1 µ  layer_metrics ķ  variables Žnon_trainable_variables layers ī regularization_losses metrics ‘layer_regularization_losses ļ trainable_variables µ ’ layer_metrics ņ  variables “non_trainable_variables ”layers ó regularization_losses •metrics –layer_regularization_losses ō trainable_variables µ — layer_metrics ö  variables ˜non_trainable_variables ™layers ÷ regularization_losses šmetrics ›layer_regularization_losses ų trainable_variables 0 „0 ¦1 §2 Ø3 ©4 Ę5  }0 ~1  }0 ~1 µ œ layer_metrics ©  variables non_trainable_variables žlayers Ŗ regularization_losses Ÿmetrics  layer_regularization_losses « trainable_variables  0 €1  0 €1 µ ” layer_metrics Æ  variables ¢non_trainable_variables £layers ° regularization_losses ¤metrics „layer_regularization_losses ± trainable_variables  0 ‚1  0 ‚1 µ ¦ layer_metrics µ  variables §non_trainable_variables Ølayers ¶ regularization_losses ©metrics Ŗlayer_regularization_losses · trainable_variables µ « layer_metrics ŗ  variables ¬non_trainable_variables ­layers » regularization_losses ®metrics Ælayer_regularization_losses ¼ trainable_variables µ ° layer_metrics ¾  variables ±non_trainable_variables ²layers æ regularization_losses ³metrics “layer_regularization_losses Ą trainable_variables 0 ×0 Ų1 Ł2 Ś3 Ū4 Ó5  0 Ž1  0 Ž1 µ µ layer_metrics ń  variables ¶non_trainable_variables ·layers ņ regularization_losses ømetrics ¹layer_regularization_losses ó trainable_variables  0 1  0 1 µ ŗ layer_metrics ÷  variables »non_trainable_variables ¼layers ų regularization_losses ½metrics ¾layer_regularization_losses ł trainable_variables  ‘0 ’1  ‘0 ’1 µ æ layer_metrics ż  variables Ąnon_trainable_variables Įlayers ž regularization_losses Āmetrics Ćlayer_regularization_losses ’ trainable_variables µ Ä layer_metrics ‚  variables Ånon_trainable_variables Ęlayers ƒ regularization_losses Ēmetrics Člayer_regularization_losses „ trainable_variables µ É layer_metrics †  variables Źnon_trainable_variables Ėlayers ‡ regularization_losses Ģmetrics Ķlayer_regularization_losses ˆ trainable_variables 0 ‰0 Š1 ‹2 Œ3 4 ą5  0 ž1  0 ž1 µ Ī layer_metrics ¹  variables Ļnon_trainable_variables Šlayers ŗ regularization_losses Ńmetrics Ņlayer_regularization_losses » trainable_variables  Ÿ0  1  Ÿ0  1 µ Ó layer_metrics æ  variables Ōnon_trainable_variables Õlayers Ą regularization_losses Ömetrics ×layer_regularization_losses Į trainable_variables  ”0 ¢1  ”0 ¢1 µ Ų layer_metrics Å  variables Łnon_trainable_variables Ślayers Ę regularization_losses Ūmetrics Ülayer_regularization_losses Ē trainable_variables µ Ż layer_metrics Ź  variables Žnon_trainable_variables ßlayers Ė regularization_losses ąmetrics įlayer_regularization_losses Ģ trainable_variables µ ā layer_metrics Ī  variables ćnon_trainable_variables älayers Ļ regularization_losses åmetrics ęlayer_regularization_losses Š trainable_variables 0 »0 ¼1 ½2 ¾3 æ4 ķ5  ­0 ®1  ­0 ®1 µ ē layer_metrics   variables čnon_trainable_variables élayers ‚ regularization_losses źmetrics ėlayer_regularization_losses ƒ trainable_variables  Æ0 °1  Æ0 °1 µ ģ layer_metrics ‡  variables ķnon_trainable_variables īlayers ˆ regularization_losses ļmetrics šlayer_regularization_losses ‰ trainable_variables  ±0 ²1  ±0 ²1 µ ń layer_metrics   variables ņnon_trainable_variables ólayers Ž regularization_losses ōmetrics õlayer_regularization_losses  trainable_variables µ ö layer_metrics ’  variables ÷non_trainable_variables ųlayers “ regularization_losses łmetrics ślayer_regularization_losses ” trainable_variables µ ū layer_metrics –  variables ünon_trainable_variables żlayers — regularization_losses žmetrics ’layer_regularization_losses ˜ trainable_variables 0 ķ0 ī1 ļ2 š3 ń4 ś5  ½0 ¾1  ½0 ¾1 µ € layer_metrics É  variables non_trainable_variables ‚layers Ź regularization_losses ƒmetrics „layer_regularization_losses Ė trainable_variables  æ0 Ą1  æ0 Ą1 µ … layer_metrics Ļ  variables †non_trainable_variables ‡layers Š regularization_losses ˆmetrics ‰layer_regularization_losses Ń trainable_variables  Į0 Ā1  Į0 Ā1 µ Š layer_metrics Õ  variables ‹non_trainable_variables Œlayers Ö regularization_losses metrics Žlayer_regularization_losses × trainable_variables µ  layer_metrics Ś  variables non_trainable_variables ‘layers Ū regularization_losses ’metrics “layer_regularization_losses Ü trainable_variables µ ” layer_metrics Ž  variables •non_trainable_variables –layers ß regularization_losses —metrics ˜layer_regularization_losses ą trainable_variables 0 Ÿ0  1 ”2 ¢3 £4 ‡5  Ķ0 Ī1  Ķ0 Ī1 µ ™ layer_metrics ‘ variables šnon_trainable_variables ›layers ’regularization_losses œmetrics layer_regularization_losses “trainable_variables  Ļ0 Š1  Ļ0 Š1 µ ž layer_metrics — variables Ÿnon_trainable_variables  layers ˜regularization_losses ”metrics ¢layer_regularization_losses ™trainable_variables  Ń0 Ņ1  Ń0 Ņ1 µ £ layer_metrics  variables ¤non_trainable_variables „layers žregularization_losses ¦metrics §layer_regularization_losses Ÿtrainable_variables µ Ø layer_metrics ¢ variables ©non_trainable_variables Ŗlayers £regularization_losses «metrics ¬layer_regularization_losses ¤trainable_variables µ ­ layer_metrics ¦ variables ®non_trainable_variables Ælayers §regularization_losses °metrics ±layer_regularization_losses Øtrainable_variables 0 Ń0 Ņ1 Ó2 Ō3 Õ4 ”5  Ż0 Ž1  Ż0 Ž1 µ ² layer_metrics Ł variables ³non_trainable_variables “layers Śregularization_losses µmetrics ¶layer_regularization_losses Ūtrainable_variables  ß0 ą1  ß0 ą1 µ · layer_metrics ß variables ønon_trainable_variables ¹layers ąregularization_losses ŗmetrics »layer_regularization_losses įtrainable_variables  į0 ā1  į0 ā1 µ ¼ layer_metrics å variables ½non_trainable_variables ¾layers ęregularization_losses æmetrics Ąlayer_regularization_losses ētrainable_variables µ Į layer_metrics ź variables Ānon_trainable_variables Ćlayers ėregularization_losses Ämetrics Ålayer_regularization_losses ģtrainable_variables µ Ę layer_metrics ī variables Ēnon_trainable_variables Člayers ļregularization_losses Émetrics Źlayer_regularization_losses štrainable_variables 0 ƒ0 „1 …2 †3 ‡4 ”5  serving_default_input_mask Placeholder*0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’* dtype0*% shape: ’’’’’’’’’ ’’’’’’’’’ “ serving_default_input_type_ids Placeholder*0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’* dtype0*% shape: ’’’’’’’’’ ’’’’’’’’’ “ serving_default_input_word_ids Placeholder*0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’* dtype0*% shape: ’’’’’’’’’ ’’’’’’’’’ ¶N StatefulPartitionedCallStatefulPartitionedCallserving_default_input_maskserving_default_input_type_idsserving_default_input_word_idsword_embeddings/embeddingsposition_embedding/embeddingstype_embeddings/embeddingsembeddings/layer_norm/gammaembeddings/layer_norm/beta/transformer/layer_0/self_attention/query/kernel-transformer/layer_0/self_attention/query/bias-transformer/layer_0/self_attention/key/kernel+transformer/layer_0/self_attention/key/bias/transformer/layer_0/self_attention/value/kernel-transformer/layer_0/self_attention/value/bias:transformer/layer_0/self_attention/attention_output/kernel8transformer/layer_0/self_attention/attention_output/bias3transformer/layer_0/self_attention_layer_norm/gamma2transformer/layer_0/self_attention_layer_norm/beta'transformer/layer_0/intermediate/kernel%transformer/layer_0/intermediate/bias!transformer/layer_0/output/kerneltransformer/layer_0/output/bias+transformer/layer_0/output_layer_norm/gamma*transformer/layer_0/output_layer_norm/beta/transformer/layer_1/self_attention/query/kernel-transformer/layer_1/self_attention/query/bias-transformer/layer_1/self_attention/key/kernel+transformer/layer_1/self_attention/key/bias/transformer/layer_1/self_attention/value/kernel-transformer/layer_1/self_attention/value/bias:transformer/layer_1/self_attention/attention_output/kernel8transformer/layer_1/self_attention/attention_output/bias3transformer/layer_1/self_attention_layer_norm/gamma2transformer/layer_1/self_attention_layer_norm/beta'transformer/layer_1/intermediate/kernel%transformer/layer_1/intermediate/bias!transformer/layer_1/output/kerneltransformer/layer_1/output/bias+transformer/layer_1/output_layer_norm/gamma*transformer/layer_1/output_layer_norm/beta/transformer/layer_2/self_attention/query/kernel-transformer/layer_2/self_attention/query/bias-transformer/layer_2/self_attention/key/kernel+transformer/layer_2/self_attention/key/bias/transformer/layer_2/self_attention/value/kernel-transformer/layer_2/self_attention/value/bias:transformer/layer_2/self_attention/attention_output/kernel8transformer/layer_2/self_attention/attention_output/bias3transformer/layer_2/self_attention_layer_norm/gamma2transformer/layer_2/self_attention_layer_norm/beta'transformer/layer_2/intermediate/kernel%transformer/layer_2/intermediate/bias!transformer/layer_2/output/kerneltransformer/layer_2/output/bias+transformer/layer_2/output_layer_norm/gamma*transformer/layer_2/output_layer_norm/beta/transformer/layer_3/self_attention/query/kernel-transformer/layer_3/self_attention/query/bias-transformer/layer_3/self_attention/key/kernel+transformer/layer_3/self_attention/key/bias/transformer/layer_3/self_attention/value/kernel-transformer/layer_3/self_attention/value/bias:transformer/layer_3/self_attention/attention_output/kernel8transformer/layer_3/self_attention/attention_output/bias3transformer/layer_3/self_attention_layer_norm/gamma2transformer/layer_3/self_attention_layer_norm/beta'transformer/layer_3/intermediate/kernel%transformer/layer_3/intermediate/bias!transformer/layer_3/output/kerneltransformer/layer_3/output/bias+transformer/layer_3/output_layer_norm/gamma*transformer/layer_3/output_layer_norm/beta/transformer/layer_4/self_attention/query/kernel-transformer/layer_4/self_attention/query/bias-transformer/layer_4/self_attention/key/kernel+transformer/layer_4/self_attention/key/bias/transformer/layer_4/self_attention/value/kernel-transformer/layer_4/self_attention/value/bias:transformer/layer_4/self_attention/attention_output/kernel8transformer/layer_4/self_attention/attention_output/bias3transformer/layer_4/self_attention_layer_norm/gamma2transformer/layer_4/self_attention_layer_norm/beta'transformer/layer_4/intermediate/kernel%transformer/layer_4/intermediate/bias!transformer/layer_4/output/kerneltransformer/layer_4/output/bias+transformer/layer_4/output_layer_norm/gamma*transformer/layer_4/output_layer_norm/beta/transformer/layer_5/self_attention/query/kernel-transformer/layer_5/self_attention/query/bias-transformer/layer_5/self_attention/key/kernel+transformer/layer_5/self_attention/key/bias/transformer/layer_5/self_attention/value/kernel-transformer/layer_5/self_attention/value/bias:transformer/layer_5/self_attention/attention_output/kernel8transformer/layer_5/self_attention/attention_output/bias3transformer/layer_5/self_attention_layer_norm/gamma2transformer/layer_5/self_attention_layer_norm/beta'transformer/layer_5/intermediate/kernel%transformer/layer_5/intermediate/bias!transformer/layer_5/output/kerneltransformer/layer_5/output/bias+transformer/layer_5/output_layer_norm/gamma*transformer/layer_5/output_layer_norm/beta/transformer/layer_6/self_attention/query/kernel-transformer/layer_6/self_attention/query/bias-transformer/layer_6/self_attention/key/kernel+transformer/layer_6/self_attention/key/bias/transformer/layer_6/self_attention/value/kernel-transformer/layer_6/self_attention/value/bias:transformer/layer_6/self_attention/attention_output/kernel8transformer/layer_6/self_attention/attention_output/bias3transformer/layer_6/self_attention_layer_norm/gamma2transformer/layer_6/self_attention_layer_norm/beta'transformer/layer_6/intermediate/kernel%transformer/layer_6/intermediate/bias!transformer/layer_6/output/kerneltransformer/layer_6/output/bias+transformer/layer_6/output_layer_norm/gamma*transformer/layer_6/output_layer_norm/beta/transformer/layer_7/self_attention/query/kernel-transformer/layer_7/self_attention/query/bias-transformer/layer_7/self_attention/key/kernel+transformer/layer_7/self_attention/key/bias/transformer/layer_7/self_attention/value/kernel-transformer/layer_7/self_attention/value/bias:transformer/layer_7/self_attention/attention_output/kernel8transformer/layer_7/self_attention/attention_output/bias3transformer/layer_7/self_attention_layer_norm/gamma2transformer/layer_7/self_attention_layer_norm/beta'transformer/layer_7/intermediate/kernel%transformer/layer_7/intermediate/bias!transformer/layer_7/output/kerneltransformer/layer_7/output/bias+transformer/layer_7/output_layer_norm/gamma*transformer/layer_7/output_layer_norm/beta/transformer/layer_8/self_attention/query/kernel-transformer/layer_8/self_attention/query/bias-transformer/layer_8/self_attention/key/kernel+transformer/layer_8/self_attention/key/bias/transformer/layer_8/self_attention/value/kernel-transformer/layer_8/self_attention/value/bias:transformer/layer_8/self_attention/attention_output/kernel8transformer/layer_8/self_attention/attention_output/bias3transformer/layer_8/self_attention_layer_norm/gamma2transformer/layer_8/self_attention_layer_norm/beta'transformer/layer_8/intermediate/kernel%transformer/layer_8/intermediate/bias!transformer/layer_8/output/kerneltransformer/layer_8/output/bias+transformer/layer_8/output_layer_norm/gamma*transformer/layer_8/output_layer_norm/beta/transformer/layer_9/self_attention/query/kernel-transformer/layer_9/self_attention/query/bias-transformer/layer_9/self_attention/key/kernel+transformer/layer_9/self_attention/key/bias/transformer/layer_9/self_attention/value/kernel-transformer/layer_9/self_attention/value/bias:transformer/layer_9/self_attention/attention_output/kernel8transformer/layer_9/self_attention/attention_output/bias3transformer/layer_9/self_attention_layer_norm/gamma2transformer/layer_9/self_attention_layer_norm/beta'transformer/layer_9/intermediate/kernel%transformer/layer_9/intermediate/bias!transformer/layer_9/output/kerneltransformer/layer_9/output/bias+transformer/layer_9/output_layer_norm/gamma*transformer/layer_9/output_layer_norm/beta0transformer/layer_10/self_attention/query/kernel.transformer/layer_10/self_attention/query/bias.transformer/layer_10/self_attention/key/kernel,transformer/layer_10/self_attention/key/bias0transformer/layer_10/self_attention/value/kernel.transformer/layer_10/self_attention/value/bias;transformer/layer_10/self_attention/attention_output/kernel9transformer/layer_10/self_attention/attention_output/bias4transformer/layer_10/self_attention_layer_norm/gamma3transformer/layer_10/self_attention_layer_norm/beta(transformer/layer_10/intermediate/kernel&transformer/layer_10/intermediate/bias"transformer/layer_10/output/kernel transformer/layer_10/output/bias,transformer/layer_10/output_layer_norm/gamma+transformer/layer_10/output_layer_norm/beta0transformer/layer_11/self_attention/query/kernel.transformer/layer_11/self_attention/query/bias.transformer/layer_11/self_attention/key/kernel,transformer/layer_11/self_attention/key/bias0transformer/layer_11/self_attention/value/kernel.transformer/layer_11/self_attention/value/bias;transformer/layer_11/self_attention/attention_output/kernel9transformer/layer_11/self_attention/attention_output/bias4transformer/layer_11/self_attention_layer_norm/gamma3transformer/layer_11/self_attention_layer_norm/beta(transformer/layer_11/intermediate/kernel&transformer/layer_11/intermediate/bias"transformer/layer_11/output/kernel transformer/layer_11/output/bias,transformer/layer_11/output_layer_norm/gamma+transformer/layer_11/output_layer_norm/betapooler_transform/kernelpooler_transform/bias*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*, f'R% #__inference_signature_wrapper_17809 O saver_filename Placeholder* _output_shapes :* dtype0* shape: Ļj StatefulPartitionedCall_1StatefulPartitionedCallsaver_filenameVariable_1/Read/ReadVariableOp.word_embeddings/embeddings/Read/ReadVariableOp1position_embedding/embeddings/Read/ReadVariableOp.type_embeddings/embeddings/Read/ReadVariableOp/embeddings/layer_norm/gamma/Read/ReadVariableOp.embeddings/layer_norm/beta/Read/ReadVariableOpCtransformer/layer_0/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_0/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_0/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_0/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_0/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_0/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_0/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_0/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_0/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_0/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_0/intermediate/kernel/Read/ReadVariableOp9transformer/layer_0/intermediate/bias/Read/ReadVariableOp5transformer/layer_0/output/kernel/Read/ReadVariableOp3transformer/layer_0/output/bias/Read/ReadVariableOp?transformer/layer_0/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_0/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_1/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_1/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_1/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_1/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_1/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_1/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_1/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_1/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_1/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_1/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_1/intermediate/kernel/Read/ReadVariableOp9transformer/layer_1/intermediate/bias/Read/ReadVariableOp5transformer/layer_1/output/kernel/Read/ReadVariableOp3transformer/layer_1/output/bias/Read/ReadVariableOp?transformer/layer_1/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_1/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_2/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_2/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_2/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_2/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_2/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_2/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_2/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_2/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_2/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_2/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_2/intermediate/kernel/Read/ReadVariableOp9transformer/layer_2/intermediate/bias/Read/ReadVariableOp5transformer/layer_2/output/kernel/Read/ReadVariableOp3transformer/layer_2/output/bias/Read/ReadVariableOp?transformer/layer_2/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_2/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_3/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_3/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_3/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_3/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_3/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_3/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_3/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_3/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_3/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_3/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_3/intermediate/kernel/Read/ReadVariableOp9transformer/layer_3/intermediate/bias/Read/ReadVariableOp5transformer/layer_3/output/kernel/Read/ReadVariableOp3transformer/layer_3/output/bias/Read/ReadVariableOp?transformer/layer_3/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_3/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_4/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_4/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_4/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_4/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_4/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_4/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_4/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_4/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_4/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_4/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_4/intermediate/kernel/Read/ReadVariableOp9transformer/layer_4/intermediate/bias/Read/ReadVariableOp5transformer/layer_4/output/kernel/Read/ReadVariableOp3transformer/layer_4/output/bias/Read/ReadVariableOp?transformer/layer_4/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_4/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_5/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_5/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_5/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_5/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_5/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_5/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_5/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_5/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_5/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_5/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_5/intermediate/kernel/Read/ReadVariableOp9transformer/layer_5/intermediate/bias/Read/ReadVariableOp5transformer/layer_5/output/kernel/Read/ReadVariableOp3transformer/layer_5/output/bias/Read/ReadVariableOp?transformer/layer_5/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_5/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_6/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_6/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_6/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_6/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_6/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_6/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_6/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_6/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_6/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_6/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_6/intermediate/kernel/Read/ReadVariableOp9transformer/layer_6/intermediate/bias/Read/ReadVariableOp5transformer/layer_6/output/kernel/Read/ReadVariableOp3transformer/layer_6/output/bias/Read/ReadVariableOp?transformer/layer_6/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_6/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_7/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_7/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_7/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_7/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_7/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_7/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_7/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_7/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_7/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_7/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_7/intermediate/kernel/Read/ReadVariableOp9transformer/layer_7/intermediate/bias/Read/ReadVariableOp5transformer/layer_7/output/kernel/Read/ReadVariableOp3transformer/layer_7/output/bias/Read/ReadVariableOp?transformer/layer_7/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_7/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_8/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_8/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_8/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_8/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_8/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_8/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_8/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_8/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_8/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_8/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_8/intermediate/kernel/Read/ReadVariableOp9transformer/layer_8/intermediate/bias/Read/ReadVariableOp5transformer/layer_8/output/kernel/Read/ReadVariableOp3transformer/layer_8/output/bias/Read/ReadVariableOp?transformer/layer_8/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_8/output_layer_norm/beta/Read/ReadVariableOpCtransformer/layer_9/self_attention/query/kernel/Read/ReadVariableOpAtransformer/layer_9/self_attention/query/bias/Read/ReadVariableOpAtransformer/layer_9/self_attention/key/kernel/Read/ReadVariableOp?transformer/layer_9/self_attention/key/bias/Read/ReadVariableOpCtransformer/layer_9/self_attention/value/kernel/Read/ReadVariableOpAtransformer/layer_9/self_attention/value/bias/Read/ReadVariableOpNtransformer/layer_9/self_attention/attention_output/kernel/Read/ReadVariableOpLtransformer/layer_9/self_attention/attention_output/bias/Read/ReadVariableOpGtransformer/layer_9/self_attention_layer_norm/gamma/Read/ReadVariableOpFtransformer/layer_9/self_attention_layer_norm/beta/Read/ReadVariableOp;transformer/layer_9/intermediate/kernel/Read/ReadVariableOp9transformer/layer_9/intermediate/bias/Read/ReadVariableOp5transformer/layer_9/output/kernel/Read/ReadVariableOp3transformer/layer_9/output/bias/Read/ReadVariableOp?transformer/layer_9/output_layer_norm/gamma/Read/ReadVariableOp>transformer/layer_9/output_layer_norm/beta/Read/ReadVariableOpDtransformer/layer_10/self_attention/query/kernel/Read/ReadVariableOpBtransformer/layer_10/self_attention/query/bias/Read/ReadVariableOpBtransformer/layer_10/self_attention/key/kernel/Read/ReadVariableOp@transformer/layer_10/self_attention/key/bias/Read/ReadVariableOpDtransformer/layer_10/self_attention/value/kernel/Read/ReadVariableOpBtransformer/layer_10/self_attention/value/bias/Read/ReadVariableOpOtransformer/layer_10/self_attention/attention_output/kernel/Read/ReadVariableOpMtransformer/layer_10/self_attention/attention_output/bias/Read/ReadVariableOpHtransformer/layer_10/self_attention_layer_norm/gamma/Read/ReadVariableOpGtransformer/layer_10/self_attention_layer_norm/beta/Read/ReadVariableOp :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_10/ExpandDims/dimų +self_attention/masked_softmax_10/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_10/ExpandDims• &self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_10/sub/x† $self_attention/masked_softmax_10/subSub/self_attention/masked_softmax_10/sub/x:output:04self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/sub• &self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_10/mul/yś $self_attention/masked_softmax_10/mulMul(self_attention/masked_softmax_10/sub:z:0/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/mulć $self_attention/masked_softmax_10/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/addÕ (self_attention/masked_softmax_10/SoftmaxSoftmax(self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_10/Softmax— 'self_attention/dropout_30/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_30/dropout/Const‡ %self_attention/dropout_30/dropout/MulMul2self_attention/masked_softmax_10/Softmax:softmax:00self_attention/dropout_30/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_30/dropout/Mul“ 'self_attention/dropout_30/dropout/ShapeShape2self_attention/masked_softmax_10/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_30/dropout/Shapeœ >self_attention/dropout_30/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_30/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_30/dropout/random_uniform/RandomUniform© 0self_attention/dropout_30/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_30/dropout/GreaterEqual/yĄ .self_attention/dropout_30/dropout/GreaterEqual GreaterEqualGself_attention/dropout_30/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_30/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_30/dropout/GreaterEqualē &self_attention/dropout_30/dropout/CastCast2self_attention/dropout_30/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_30/dropout/Castü 'self_attention/dropout_30/dropout/Mul_1Mul)self_attention/dropout_30/dropout/Mul:z:0*self_attention/dropout_30/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_30/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_30/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_31/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_31/dropout/ConstĆ dropout_31/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_31/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/dropout/Mul‹ dropout_31/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_31/dropout/Shapeć /dropout_31/dropout/random_uniform/RandomUniform RandomUniform!dropout_31/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_31/dropout/random_uniform/RandomUniform‹ !dropout_31/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_31/dropout/GreaterEqual/yų dropout_31/dropout/GreaterEqual GreaterEqual8dropout_31/dropout/random_uniform/RandomUniform:output:0*dropout_31/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_31/dropout/GreaterEqual® dropout_31/dropout/CastCast#dropout_31/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/dropout/Cast“ dropout_31/dropout/Mul_1Muldropout_31/dropout/Mul:z:0dropout_31/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/dropout/Mul_1y addAddV2inputsdropout_31/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_10/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_10/Pow/y” activation_10/PowPowintermediate/add:z:0activation_10/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Powo activation_10/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_10/mul/x¢ activation_10/mulMulactivation_10/mul/x:output:0activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mulœ activation_10/addAddV2intermediate/add:z:0activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/adds activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_10/mul_1/xØ activation_10/mul_1Mulactivation_10/mul_1/x:output:0activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_1‰ activation_10/TanhTanhactivation_10/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Tanhs activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_10/add_1/x« activation_10/add_1AddV2activation_10/add_1/x:output:0activation_10/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/add_1s activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_10/mul_2/xŖ activation_10/mul_2Mulactivation_10/mul_2/x:output:0activation_10/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_2  activation_10/mul_3Mulintermediate/add:z:0activation_10/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_10/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_32/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_32/dropout/ConstŖ dropout_32/dropout/MulMuloutput/add:z:0!dropout_32/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/dropout/Mulr dropout_32/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_32/dropout/Shapeć /dropout_32/dropout/random_uniform/RandomUniform RandomUniform!dropout_32/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_32/dropout/random_uniform/RandomUniform‹ !dropout_32/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_32/dropout/GreaterEqual/yų dropout_32/dropout/GreaterEqual GreaterEqual8dropout_32/dropout/random_uniform/RandomUniform:output:0*dropout_32/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_32/dropout/GreaterEqual® dropout_32/dropout/CastCast#dropout_32/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/dropout/Cast“ dropout_32/dropout/Mul_1Muldropout_32/dropout/Mul:z:0dropout_32/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/dropout/Mul_1¤ add_1AddV2dropout_32/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : К ¢ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_11878 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_11/ExpandDims/dimų +self_attention/masked_softmax_11/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_11/ExpandDims• &self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_11/sub/x† $self_attention/masked_softmax_11/subSub/self_attention/masked_softmax_11/sub/x:output:04self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/sub• &self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_11/mul/yś $self_attention/masked_softmax_11/mulMul(self_attention/masked_softmax_11/sub:z:0/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/mulć $self_attention/masked_softmax_11/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/addÕ (self_attention/masked_softmax_11/SoftmaxSoftmax(self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_11/SoftmaxŌ "self_attention/dropout_33/IdentityIdentity2self_attention/masked_softmax_11/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_33/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_33/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_34/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/Identityy addAddV2inputsdropout_34/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_11/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_11/Pow/y” activation_11/PowPowintermediate/add:z:0activation_11/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Powo activation_11/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_11/mul/x¢ activation_11/mulMulactivation_11/mul/x:output:0activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mulœ activation_11/addAddV2intermediate/add:z:0activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/adds activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_11/mul_1/xØ activation_11/mul_1Mulactivation_11/mul_1/x:output:0activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_1‰ activation_11/TanhTanhactivation_11/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Tanhs activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_11/add_1/x« activation_11/add_1AddV2activation_11/add_1/x:output:0activation_11/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/add_1s activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_11/mul_2/xŖ activation_11/mul_2Mulactivation_11/mul_2/x:output:0activation_11/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_2  activation_11/mul_3Mulintermediate/add:z:0activation_11/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_11/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_35/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/Identity¤ add_1AddV2dropout_35/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ź € J__inference_word_embeddings_layer_call_and_return_conditional_losses_25325 inputs gather_resource identity_1ˆq Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’2 Reshape/shapek ReshapeReshapeinputsReshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2 ReshapeŒ GatherResourceGathergather_resourceReshape:output:0* Tindices0*( _output_shapes : ’’’’’’’’’€* dtype02 Gatherd IdentityIdentityGather:output:0* T0*( _output_shapes : ’’’’’’’’’€2 IdentityD ShapeShapeinputs* T0* _output_shapes :2 Shapem concat/values_1Const* _output_shapes :* dtype0* valueB :€2 concat/values_1\ concat/axisConst* _output_shapes :* dtype0* value B:2 concat/axisŠ concatConcatV2Shape:output:0concat/values_1:output:0concat/axis:output:0* N* T0* _output_shapes :2 concat… Reshape_1ReshapeIdentity:output:0concat:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Reshape_1x Identity_1IdentityReshape_1:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1"! identity_1Identity_1:output:0*3 _input_shapes" : ’’’’’’’’’ ’’’’’’’’’::XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes : ×ģ Ļ‘ !__inference__traced_restore_30529 file_prefix assignvariableop_variable_11 -assignvariableop_1_word_embeddings_embeddings4 0assignvariableop_2_position_embedding_embeddings1 -assignvariableop_3_type_embeddings_embeddings2 .assignvariableop_4_embeddings_layer_norm_gamma1 -assignvariableop_5_embeddings_layer_norm_betaF Bassignvariableop_6_transformer_layer_0_self_attention_query_kernelD @assignvariableop_7_transformer_layer_0_self_attention_query_biasD @assignvariableop_8_transformer_layer_0_self_attention_key_kernelB >assignvariableop_9_transformer_layer_0_self_attention_key_biasG Cassignvariableop_10_transformer_layer_0_self_attention_value_kernelE Aassignvariableop_11_transformer_layer_0_self_attention_value_biasR Nassignvariableop_12_transformer_layer_0_self_attention_attention_output_kernelP Lassignvariableop_13_transformer_layer_0_self_attention_attention_output_biasK Gassignvariableop_14_transformer_layer_0_self_attention_layer_norm_gammaJ Fassignvariableop_15_transformer_layer_0_self_attention_layer_norm_beta? ;assignvariableop_16_transformer_layer_0_intermediate_kernel= 9assignvariableop_17_transformer_layer_0_intermediate_bias9 5assignvariableop_18_transformer_layer_0_output_kernel7 3assignvariableop_19_transformer_layer_0_output_biasC ?assignvariableop_20_transformer_layer_0_output_layer_norm_gammaB >assignvariableop_21_transformer_layer_0_output_layer_norm_betaG Cassignvariableop_22_transformer_layer_1_self_attention_query_kernelE Aassignvariableop_23_transformer_layer_1_self_attention_query_biasE Aassignvariableop_24_transformer_layer_1_self_attention_key_kernelC ?assignvariableop_25_transformer_layer_1_self_attention_key_biasG Cassignvariableop_26_transformer_layer_1_self_attention_value_kernelE Aassignvariableop_27_transformer_layer_1_self_attention_value_biasR Nassignvariableop_28_transformer_layer_1_self_attention_attention_output_kernelP Lassignvariableop_29_transformer_layer_1_self_attention_attention_output_biasK Gassignvariableop_30_transformer_layer_1_self_attention_layer_norm_gammaJ Fassignvariableop_31_transformer_layer_1_self_attention_layer_norm_beta? ;assignvariableop_32_transformer_layer_1_intermediate_kernel= 9assignvariableop_33_transformer_layer_1_intermediate_bias9 5assignvariableop_34_transformer_layer_1_output_kernel7 3assignvariableop_35_transformer_layer_1_output_biasC ?assignvariableop_36_transformer_layer_1_output_layer_norm_gammaB >assignvariableop_37_transformer_layer_1_output_layer_norm_betaG Cassignvariableop_38_transformer_layer_2_self_attention_query_kernelE Aassignvariableop_39_transformer_layer_2_self_attention_query_biasE Aassignvariableop_40_transformer_layer_2_self_attention_key_kernelC ?assignvariableop_41_transformer_layer_2_self_attention_key_biasG Cassignvariableop_42_transformer_layer_2_self_attention_value_kernelE Aassignvariableop_43_transformer_layer_2_self_attention_value_biasR Nassignvariableop_44_transformer_layer_2_self_attention_attention_output_kernelP Lassignvariableop_45_transformer_layer_2_self_attention_attention_output_biasK Gassignvariableop_46_transformer_layer_2_self_attention_layer_norm_gammaJ Fassignvariableop_47_transformer_layer_2_self_attention_layer_norm_beta? ;assignvariableop_48_transformer_layer_2_intermediate_kernel= 9assignvariableop_49_transformer_layer_2_intermediate_bias9 5assignvariableop_50_transformer_layer_2_output_kernel7 3assignvariableop_51_transformer_layer_2_output_biasC ?assignvariableop_52_transformer_layer_2_output_layer_norm_gammaB >assignvariableop_53_transformer_layer_2_output_layer_norm_betaG Cassignvariableop_54_transformer_layer_3_self_attention_query_kernelE Aassignvariableop_55_transformer_layer_3_self_attention_query_biasE Aassignvariableop_56_transformer_layer_3_self_attention_key_kernelC ?assignvariableop_57_transformer_layer_3_self_attention_key_biasG Cassignvariableop_58_transformer_layer_3_self_attention_value_kernelE Aassignvariableop_59_transformer_layer_3_self_attention_value_biasR Nassignvariableop_60_transformer_layer_3_self_attention_attention_output_kernelP Lassignvariableop_61_transformer_layer_3_self_attention_attention_output_biasK Gassignvariableop_62_transformer_layer_3_self_attention_layer_norm_gammaJ Fassignvariableop_63_transformer_layer_3_self_attention_layer_norm_beta? ;assignvariableop_64_transformer_layer_3_intermediate_kernel= 9assignvariableop_65_transformer_layer_3_intermediate_bias9 5assignvariableop_66_transformer_layer_3_output_kernel7 3assignvariableop_67_transformer_layer_3_output_biasC ?assignvariableop_68_transformer_layer_3_output_layer_norm_gammaB >assignvariableop_69_transformer_layer_3_output_layer_norm_betaG Cassignvariableop_70_transformer_layer_4_self_attention_query_kernelE Aassignvariableop_71_transformer_layer_4_self_attention_query_biasE Aassignvariableop_72_transformer_layer_4_self_attention_key_kernelC ?assignvariableop_73_transformer_layer_4_self_attention_key_biasG Cassignvariableop_74_transformer_layer_4_self_attention_value_kernelE Aassignvariableop_75_transformer_layer_4_self_attention_value_biasR Nassignvariableop_76_transformer_layer_4_self_attention_attention_output_kernelP Lassignvariableop_77_transformer_layer_4_self_attention_attention_output_biasK Gassignvariableop_78_transformer_layer_4_self_attention_layer_norm_gammaJ Fassignvariableop_79_transformer_layer_4_self_attention_layer_norm_beta? ;assignvariableop_80_transformer_layer_4_intermediate_kernel= 9assignvariableop_81_transformer_layer_4_intermediate_bias9 5assignvariableop_82_transformer_layer_4_output_kernel7 3assignvariableop_83_transformer_layer_4_output_biasC ?assignvariableop_84_transformer_layer_4_output_layer_norm_gammaB >assignvariableop_85_transformer_layer_4_output_layer_norm_betaG Cassignvariableop_86_transformer_layer_5_self_attention_query_kernelE Aassignvariableop_87_transformer_layer_5_self_attention_query_biasE Aassignvariableop_88_transformer_layer_5_self_attention_key_kernelC ?assignvariableop_89_transformer_layer_5_self_attention_key_biasG Cassignvariableop_90_transformer_layer_5_self_attention_value_kernelE Aassignvariableop_91_transformer_layer_5_self_attention_value_biasR Nassignvariableop_92_transformer_layer_5_self_attention_attention_output_kernelP Lassignvariableop_93_transformer_layer_5_self_attention_attention_output_biasK Gassignvariableop_94_transformer_layer_5_self_attention_layer_norm_gammaJ Fassignvariableop_95_transformer_layer_5_self_attention_layer_norm_beta? ;assignvariableop_96_transformer_layer_5_intermediate_kernel= 9assignvariableop_97_transformer_layer_5_intermediate_bias9 5assignvariableop_98_transformer_layer_5_output_kernel7 3assignvariableop_99_transformer_layer_5_output_biasD @assignvariableop_100_transformer_layer_5_output_layer_norm_gammaC ?assignvariableop_101_transformer_layer_5_output_layer_norm_betaH Dassignvariableop_102_transformer_layer_6_self_attention_query_kernelF Bassignvariableop_103_transformer_layer_6_self_attention_query_biasF Bassignvariableop_104_transformer_layer_6_self_attention_key_kernelD @assignvariableop_105_transformer_layer_6_self_attention_key_biasH Dassignvariableop_106_transformer_layer_6_self_attention_value_kernelF Bassignvariableop_107_transformer_layer_6_self_attention_value_biasS Oassignvariableop_108_transformer_layer_6_self_attention_attention_output_kernelQ Massignvariableop_109_transformer_layer_6_self_attention_attention_output_biasL Hassignvariableop_110_transformer_layer_6_self_attention_layer_norm_gammaK Gassignvariableop_111_transformer_layer_6_self_attention_layer_norm_beta@ :assignvariableop_113_transformer_layer_6_intermediate_bias: 6assignvariableop_114_transformer_layer_6_output_kernel8 4assignvariableop_115_transformer_layer_6_output_biasD @assignvariableop_116_transformer_layer_6_output_layer_norm_gammaC ?assignvariableop_117_transformer_layer_6_output_layer_norm_betaH Dassignvariableop_118_transformer_layer_7_self_attention_query_kernelF Bassignvariableop_119_transformer_layer_7_self_attention_query_biasF Bassignvariableop_120_transformer_layer_7_self_attention_key_kernelD @assignvariableop_121_transformer_layer_7_self_attention_key_biasH Dassignvariableop_122_transformer_layer_7_self_attention_value_kernelF Bassignvariableop_123_transformer_layer_7_self_attention_value_biasS Oassignvariableop_124_transformer_layer_7_self_attention_attention_output_kernelQ Massignvariableop_125_transformer_layer_7_self_attention_attention_output_biasL Hassignvariableop_126_transformer_layer_7_self_attention_layer_norm_gammaK Gassignvariableop_127_transformer_layer_7_self_attention_layer_norm_beta@ :assignvariableop_129_transformer_layer_7_intermediate_bias: 6assignvariableop_130_transformer_layer_7_output_kernel8 4assignvariableop_131_transformer_layer_7_output_biasD @assignvariableop_132_transformer_layer_7_output_layer_norm_gammaC ?assignvariableop_133_transformer_layer_7_output_layer_norm_betaH Dassignvariableop_134_transformer_layer_8_self_attention_query_kernelF Bassignvariableop_135_transformer_layer_8_self_attention_query_biasF Bassignvariableop_136_transformer_layer_8_self_attention_key_kernelD @assignvariableop_137_transformer_layer_8_self_attention_key_biasH Dassignvariableop_138_transformer_layer_8_self_attention_value_kernelF Bassignvariableop_139_transformer_layer_8_self_attention_value_biasS Oassignvariableop_140_transformer_layer_8_self_attention_attention_output_kernelQ Massignvariableop_141_transformer_layer_8_self_attention_attention_output_biasL Hassignvariableop_142_transformer_layer_8_self_attention_layer_norm_gammaK Gassignvariableop_143_transformer_layer_8_self_attention_layer_norm_beta@ :assignvariableop_145_transformer_layer_8_intermediate_bias: 6assignvariableop_146_transformer_layer_8_output_kernel8 4assignvariableop_147_transformer_layer_8_output_biasD @assignvariableop_148_transformer_layer_8_output_layer_norm_gammaC ?assignvariableop_149_transformer_layer_8_output_layer_norm_betaH Dassignvariableop_150_transformer_layer_9_self_attention_query_kernelF Bassignvariableop_151_transformer_layer_9_self_attention_query_biasF Bassignvariableop_152_transformer_layer_9_self_attention_key_kernelD @assignvariableop_153_transformer_layer_9_self_attention_key_biasH Dassignvariableop_154_transformer_layer_9_self_attention_value_kernelF Bassignvariableop_155_transformer_layer_9_self_attention_value_biasS Oassignvariableop_156_transformer_layer_9_self_attention_attention_output_kernelQ Massignvariableop_157_transformer_layer_9_self_attention_attention_output_biasL Hassignvariableop_158_transformer_layer_9_self_attention_layer_norm_gammaK Gassignvariableop_159_transformer_layer_9_self_attention_layer_norm_beta@ :assignvariableop_161_transformer_layer_9_intermediate_bias: 6assignvariableop_162_transformer_layer_9_output_kernel8 4assignvariableop_163_transformer_layer_9_output_biasD @assignvariableop_164_transformer_layer_9_output_layer_norm_gammaC ?assignvariableop_165_transformer_layer_9_output_layer_norm_betaI Eassignvariableop_166_transformer_layer_10_self_attention_query_kernelG Cassignvariableop_167_transformer_layer_10_self_attention_query_biasG Cassignvariableop_168_transformer_layer_10_self_attention_key_kernelE Aassignvariableop_169_transformer_layer_10_self_attention_key_biasI Eassignvariableop_170_transformer_layer_10_self_attention_value_kernelG Cassignvariableop_171_transformer_layer_10_self_attention_value_biasT Passignvariableop_172_transformer_layer_10_self_attention_attention_output_kernelR Nassignvariableop_173_transformer_layer_10_self_attention_attention_output_biasM Iassignvariableop_174_transformer_layer_10_self_attention_layer_norm_gammaL Hassignvariableop_175_transformer_layer_10_self_attention_layer_norm_betaA =assignvariableop_176_transformer_layer_10_intermediate_kernel? ;assignvariableop_177_transformer_layer_10_intermediate_bias; 7assignvariableop_178_transformer_layer_10_output_kernel9 5assignvariableop_179_transformer_layer_10_output_biasE Aassignvariableop_180_transformer_layer_10_output_layer_norm_gammaD @assignvariableop_181_transformer_layer_10_output_layer_norm_betaI Eassignvariableop_182_transformer_layer_11_self_attention_query_kernelG Cassignvariableop_183_transformer_layer_11_self_attention_query_biasG Cassignvariableop_184_transformer_layer_11_self_attention_key_kernelE Aassignvariableop_185_transformer_layer_11_self_attention_key_biasI Eassignvariableop_186_transformer_layer_11_self_attention_value_kernelG Cassignvariableop_187_transformer_layer_11_self_attention_value_biasT Passignvariableop_188_transformer_layer_11_self_attention_attention_output_kernelR Nassignvariableop_189_transformer_layer_11_self_attention_attention_output_biasM Iassignvariableop_190_transformer_layer_11_self_attention_layer_norm_gammaL Hassignvariableop_191_transformer_layer_11_self_attention_layer_norm_betaA =assignvariableop_192_transformer_layer_11_intermediate_kernel? ;assignvariableop_193_transformer_layer_11_intermediate_bias; 7assignvariableop_194_transformer_layer_11_output_kernel9 5assignvariableop_195_transformer_layer_11_output_biasE Aassignvariableop_196_transformer_layer_11_output_layer_norm_gammaD @assignvariableop_197_transformer_layer_11_output_layer_norm_beta0 ,assignvariableop_198_pooler_transform_kernel. *assignvariableop_199_pooler_transform_bias identity_201ˆ¢AssignVariableOp¢AssignVariableOp_1¢AssignVariableOp_10¢AssignVariableOp_100¢AssignVariableOp_101¢AssignVariableOp_102¢AssignVariableOp_103¢AssignVariableOp_104¢AssignVariableOp_105¢AssignVariableOp_106¢AssignVariableOp_107¢AssignVariableOp_108¢AssignVariableOp_109¢AssignVariableOp_11¢AssignVariableOp_110¢AssignVariableOp_111¢AssignVariableOp_112¢AssignVariableOp_113¢AssignVariableOp_114¢AssignVariableOp_115¢AssignVariableOp_116¢AssignVariableOp_117¢AssignVariableOp_118¢AssignVariableOp_119¢AssignVariableOp_12¢AssignVariableOp_120¢AssignVariableOp_121¢AssignVariableOp_122¢AssignVariableOp_123¢AssignVariableOp_124¢AssignVariableOp_125¢AssignVariableOp_126¢AssignVariableOp_127¢AssignVariableOp_128¢AssignVariableOp_129¢AssignVariableOp_13¢AssignVariableOp_130¢AssignVariableOp_131¢AssignVariableOp_132¢AssignVariableOp_133¢AssignVariableOp_134¢AssignVariableOp_135¢AssignVariableOp_136¢AssignVariableOp_137¢AssignVariableOp_138¢AssignVariableOp_139¢AssignVariableOp_14¢AssignVariableOp_140¢AssignVariableOp_141¢AssignVariableOp_142¢AssignVariableOp_143¢AssignVariableOp_144¢AssignVariableOp_145¢AssignVariableOp_146¢AssignVariableOp_147¢AssignVariableOp_148¢AssignVariableOp_149¢AssignVariableOp_15¢AssignVariableOp_150¢AssignVariableOp_151¢AssignVariableOp_152¢AssignVariableOp_153¢AssignVariableOp_154¢AssignVariableOp_155¢AssignVariableOp_156¢AssignVariableOp_157¢AssignVariableOp_158¢AssignVariableOp_159¢AssignVariableOp_16¢AssignVariableOp_160¢AssignVariableOp_161¢AssignVariableOp_162¢AssignVariableOp_163¢AssignVariableOp_164¢AssignVariableOp_165¢AssignVariableOp_166¢AssignVariableOp_167¢AssignVariableOp_168¢AssignVariableOp_169¢AssignVariableOp_17¢AssignVariableOp_170¢AssignVariableOp_171¢AssignVariableOp_172¢AssignVariableOp_173¢AssignVariableOp_174¢AssignVariableOp_175¢AssignVariableOp_176¢AssignVariableOp_177¢AssignVariableOp_178¢AssignVariableOp_179¢AssignVariableOp_18¢AssignVariableOp_180¢AssignVariableOp_181¢AssignVariableOp_182¢AssignVariableOp_183¢AssignVariableOp_184¢AssignVariableOp_185¢AssignVariableOp_186¢AssignVariableOp_187¢AssignVariableOp_188¢AssignVariableOp_189¢AssignVariableOp_19¢AssignVariableOp_190¢AssignVariableOp_191¢AssignVariableOp_192¢AssignVariableOp_193¢AssignVariableOp_194¢AssignVariableOp_195¢AssignVariableOp_196¢AssignVariableOp_197¢AssignVariableOp_198¢AssignVariableOp_199¢AssignVariableOp_2¢AssignVariableOp_20¢AssignVariableOp_21¢AssignVariableOp_22¢AssignVariableOp_23¢AssignVariableOp_24¢AssignVariableOp_25¢AssignVariableOp_26¢AssignVariableOp_27¢AssignVariableOp_28¢AssignVariableOp_29¢AssignVariableOp_3¢AssignVariableOp_30¢AssignVariableOp_31¢AssignVariableOp_32¢AssignVariableOp_33¢AssignVariableOp_34¢AssignVariableOp_35¢AssignVariableOp_36¢AssignVariableOp_37¢AssignVariableOp_38¢AssignVariableOp_39¢AssignVariableOp_4¢AssignVariableOp_40¢AssignVariableOp_41¢AssignVariableOp_42¢AssignVariableOp_43¢AssignVariableOp_44¢AssignVariableOp_45¢AssignVariableOp_46¢AssignVariableOp_47¢AssignVariableOp_48¢AssignVariableOp_49¢AssignVariableOp_5¢AssignVariableOp_50¢AssignVariableOp_51¢AssignVariableOp_52¢AssignVariableOp_53¢AssignVariableOp_54¢AssignVariableOp_55¢AssignVariableOp_56¢AssignVariableOp_57¢AssignVariableOp_58¢AssignVariableOp_59¢AssignVariableOp_6¢AssignVariableOp_60¢AssignVariableOp_61¢AssignVariableOp_62¢AssignVariableOp_63¢AssignVariableOp_64¢AssignVariableOp_65¢AssignVariableOp_66¢AssignVariableOp_67¢AssignVariableOp_68¢AssignVariableOp_69¢AssignVariableOp_7¢AssignVariableOp_70¢AssignVariableOp_71¢AssignVariableOp_72¢AssignVariableOp_73¢AssignVariableOp_74¢AssignVariableOp_75¢AssignVariableOp_76¢AssignVariableOp_77¢AssignVariableOp_78¢AssignVariableOp_79¢AssignVariableOp_8¢AssignVariableOp_80¢AssignVariableOp_81¢AssignVariableOp_82¢AssignVariableOp_83¢AssignVariableOp_84¢AssignVariableOp_85¢AssignVariableOp_86¢AssignVariableOp_87¢AssignVariableOp_88¢AssignVariableOp_89¢AssignVariableOp_9¢AssignVariableOp_90¢AssignVariableOp_91¢AssignVariableOp_92¢AssignVariableOp_93¢AssignVariableOp_94¢AssignVariableOp_95¢AssignVariableOp_96¢AssignVariableOp_97¢AssignVariableOp_98¢AssignVariableOp_99¢ RestoreV2¢ RestoreV2_1ķA RestoreV2/tensor_namesConst" /device:CPU:0* _output_shapes :Č* dtype0*ų@ valueī@Bė@ČB(do_lower_case/.ATTRIBUTES/VARIABLE_VALUEB&variables/0/.ATTRIBUTES/VARIABLE_VALUEB&variables/1/.ATTRIBUTES/VARIABLE_VALUEB&variables/2/.ATTRIBUTES/VARIABLE_VALUEB&variables/3/.ATTRIBUTES/VARIABLE_VALUEB&variables/4/.ATTRIBUTES/VARIABLE_VALUEB&variables/5/.ATTRIBUTES/VARIABLE_VALUEB&variables/6/.ATTRIBUTES/VARIABLE_VALUEB&variables/7/.ATTRIBUTES/VARIABLE_VALUEB&variables/8/.ATTRIBUTES/VARIABLE_VALUEB&variables/9/.ATTRIBUTES/VARIABLE_VALUEB'variables/10/.ATTRIBUTES/VARIABLE_VALUEB'variables/11/.ATTRIBUTES/VARIABLE_VALUEB'variables/12/.ATTRIBUTES/VARIABLE_VALUEB'variables/13/.ATTRIBUTES/VARIABLE_VALUEB'variables/14/.ATTRIBUTES/VARIABLE_VALUEB'variables/15/.ATTRIBUTES/VARIABLE_VALUEB'variables/16/.ATTRIBUTES/VARIABLE_VALUEB'variables/17/.ATTRIBUTES/VARIABLE_VALUEB'variables/18/.ATTRIBUTES/VARIABLE_VALUEB'variables/19/.ATTRIBUTES/VARIABLE_VALUEB'variables/20/.ATTRIBUTES/VARIABLE_VALUEB'variables/21/.ATTRIBUTES/VARIABLE_VALUEB'variables/22/.ATTRIBUTES/VARIABLE_VALUEB'variables/23/.ATTRIBUTES/VARIABLE_VALUEB'variables/24/.ATTRIBUTES/VARIABLE_VALUEB'variables/25/.ATTRIBUTES/VARIABLE_VALUEB'variables/26/.ATTRIBUTES/VARIABLE_VALUEB'variables/27/.ATTRIBUTES/VARIABLE_VALUEB'variables/28/.ATTRIBUTES/VARIABLE_VALUEB'variables/29/.ATTRIBUTES/VARIABLE_VALUEB'variables/30/.ATTRIBUTES/VARIABLE_VALUEB'variables/31/.ATTRIBUTES/VARIABLE_VALUEB'variables/32/.ATTRIBUTES/VARIABLE_VALUEB'variables/33/.ATTRIBUTES/VARIABLE_VALUEB'variables/34/.ATTRIBUTES/VARIABLE_VALUEB'variables/35/.ATTRIBUTES/VARIABLE_VALUEB'variables/36/.ATTRIBUTES/VARIABLE_VALUEB'variables/37/.ATTRIBUTES/VARIABLE_VALUEB'variables/38/.ATTRIBUTES/VARIABLE_VALUEB'variables/39/.ATTRIBUTES/VARIABLE_VALUEB'variables/40/.ATTRIBUTES/VARIABLE_VALUEB'variables/41/.ATTRIBUTES/VARIABLE_VALUEB'variables/42/.ATTRIBUTES/VARIABLE_VALUEB'variables/43/.ATTRIBUTES/VARIABLE_VALUEB'variables/44/.ATTRIBUTES/VARIABLE_VALUEB'variables/45/.ATTRIBUTES/VARIABLE_VALUEB'variables/46/.ATTRIBUTES/VARIABLE_VALUEB'variables/47/.ATTRIBUTES/VARIABLE_VALUEB'variables/48/.ATTRIBUTES/VARIABLE_VALUEB'variables/49/.ATTRIBUTES/VARIABLE_VALUEB'variables/50/.ATTRIBUTES/VARIABLE_VALUEB'variables/51/.ATTRIBUTES/VARIABLE_VALUEB'variables/52/.ATTRIBUTES/VARIABLE_VALUEB'variables/53/.ATTRIBUTES/VARIABLE_VALUEB'variables/54/.ATTRIBUTES/VARIABLE_VALUEB'variables/55/.ATTRIBUTES/VARIABLE_VALUEB'variables/56/.ATTRIBUTES/VARIABLE_VALUEB'variables/57/.ATTRIBUTES/VARIABLE_VALUEB'variables/58/.ATTRIBUTES/VARIABLE_VALUEB'variables/59/.ATTRIBUTES/VARIABLE_VALUEB'variables/60/.ATTRIBUTES/VARIABLE_VALUEB'variables/61/.ATTRIBUTES/VARIABLE_VALUEB'variables/62/.ATTRIBUTES/VARIABLE_VALUEB'variables/63/.ATTRIBUTES/VARIABLE_VALUEB'variables/64/.ATTRIBUTES/VARIABLE_VALUEB'variables/65/.ATTRIBUTES/VARIABLE_VALUEB'variables/66/.ATTRIBUTES/VARIABLE_VALUEB'variables/67/.ATTRIBUTES/VARIABLE_VALUEB'variables/68/.ATTRIBUTES/VARIABLE_VALUEB'variables/69/.ATTRIBUTES/VARIABLE_VALUEB'variables/70/.ATTRIBUTES/VARIABLE_VALUEB'variables/71/.ATTRIBUTES/VARIABLE_VALUEB'variables/72/.ATTRIBUTES/VARIABLE_VALUEB'variables/73/.ATTRIBUTES/VARIABLE_VALUEB'variables/74/.ATTRIBUTES/VARIABLE_VALUEB'variables/75/.ATTRIBUTES/VARIABLE_VALUEB'variables/76/.ATTRIBUTES/VARIABLE_VALUEB'variables/77/.ATTRIBUTES/VARIABLE_VALUEB'variables/78/.ATTRIBUTES/VARIABLE_VALUEB'variables/79/.ATTRIBUTES/VARIABLE_VALUEB'variables/80/.ATTRIBUTES/VARIABLE_VALUEB'variables/81/.ATTRIBUTES/VARIABLE_VALUEB'variables/82/.ATTRIBUTES/VARIABLE_VALUEB'variables/83/.ATTRIBUTES/VARIABLE_VALUEB'variables/84/.ATTRIBUTES/VARIABLE_VALUEB'variables/85/.ATTRIBUTES/VARIABLE_VALUEB'variables/86/.ATTRIBUTES/VARIABLE_VALUEB'variables/87/.ATTRIBUTES/VARIABLE_VALUEB'variables/88/.ATTRIBUTES/VARIABLE_VALUEB'variables/89/.ATTRIBUTES/VARIABLE_VALUEB'variables/90/.ATTRIBUTES/VARIABLE_VALUEB'variables/91/.ATTRIBUTES/VARIABLE_VALUEB'variables/92/.ATTRIBUTES/VARIABLE_VALUEB'variables/93/.ATTRIBUTES/VARIABLE_VALUEB'variables/94/.ATTRIBUTES/VARIABLE_VALUEB'variables/95/.ATTRIBUTES/VARIABLE_VALUEB'variables/96/.ATTRIBUTES/VARIABLE_VALUEB'variables/97/.ATTRIBUTES/VARIABLE_VALUEB'variables/98/.ATTRIBUTES/VARIABLE_VALUEB'variables/99/.ATTRIBUTES/VARIABLE_VALUEB(variables/100/.ATTRIBUTES/VARIABLE_VALUEB(variables/101/.ATTRIBUTES/VARIABLE_VALUEB(variables/102/.ATTRIBUTES/VARIABLE_VALUEB(variables/103/.ATTRIBUTES/VARIABLE_VALUEB(variables/104/.ATTRIBUTES/VARIABLE_VALUEB(variables/105/.ATTRIBUTES/VARIABLE_VALUEB(variables/106/.ATTRIBUTES/VARIABLE_VALUEB(variables/107/.ATTRIBUTES/VARIABLE_VALUEB(variables/108/.ATTRIBUTES/VARIABLE_VALUEB(variables/109/.ATTRIBUTES/VARIABLE_VALUEB(variables/110/.ATTRIBUTES/VARIABLE_VALUEB(variables/111/.ATTRIBUTES/VARIABLE_VALUEB(variables/112/.ATTRIBUTES/VARIABLE_VALUEB(variables/113/.ATTRIBUTES/VARIABLE_VALUEB(variables/114/.ATTRIBUTES/VARIABLE_VALUEB(variables/115/.ATTRIBUTES/VARIABLE_VALUEB(variables/116/.ATTRIBUTES/VARIABLE_VALUEB(variables/117/.ATTRIBUTES/VARIABLE_VALUEB(variables/118/.ATTRIBUTES/VARIABLE_VALUEB(variables/119/.ATTRIBUTES/VARIABLE_VALUEB(variables/120/.ATTRIBUTES/VARIABLE_VALUEB(variables/121/.ATTRIBUTES/VARIABLE_VALUEB(variables/122/.ATTRIBUTES/VARIABLE_VALUEB(variables/123/.ATTRIBUTES/VARIABLE_VALUEB(variables/124/.ATTRIBUTES/VARIABLE_VALUEB(variables/125/.ATTRIBUTES/VARIABLE_VALUEB(variables/126/.ATTRIBUTES/VARIABLE_VALUEB(variables/127/.ATTRIBUTES/VARIABLE_VALUEB(variables/128/.ATTRIBUTES/VARIABLE_VALUEB(variables/129/.ATTRIBUTES/VARIABLE_VALUEB(variables/130/.ATTRIBUTES/VARIABLE_VALUEB(variables/131/.ATTRIBUTES/VARIABLE_VALUEB(variables/132/.ATTRIBUTES/VARIABLE_VALUEB(variables/133/.ATTRIBUTES/VARIABLE_VALUEB(variables/134/.ATTRIBUTES/VARIABLE_VALUEB(variables/135/.ATTRIBUTES/VARIABLE_VALUEB(variables/136/.ATTRIBUTES/VARIABLE_VALUEB(variables/137/.ATTRIBUTES/VARIABLE_VALUEB(variables/138/.ATTRIBUTES/VARIABLE_VALUEB(variables/139/.ATTRIBUTES/VARIABLE_VALUEB(variables/140/.ATTRIBUTES/VARIABLE_VALUEB(variables/141/.ATTRIBUTES/VARIABLE_VALUEB(variables/142/.ATTRIBUTES/VARIABLE_VALUEB(variables/143/.ATTRIBUTES/VARIABLE_VALUEB(variables/144/.ATTRIBUTES/VARIABLE_VALUEB(variables/145/.ATTRIBUTES/VARIABLE_VALUEB(variables/146/.ATTRIBUTES/VARIABLE_VALUEB(variables/147/.ATTRIBUTES/VARIABLE_VALUEB(variables/148/.ATTRIBUTES/VARIABLE_VALUEB(variables/149/.ATTRIBUTES/VARIABLE_VALUEB(variables/150/.ATTRIBUTES/VARIABLE_VALUEB(variables/151/.ATTRIBUTES/VARIABLE_VALUEB(variables/152/.ATTRIBUTES/VARIABLE_VALUEB(variables/153/.ATTRIBUTES/VARIABLE_VALUEB(variables/154/.ATTRIBUTES/VARIABLE_VALUEB(variables/155/.ATTRIBUTES/VARIABLE_VALUEB(variables/156/.ATTRIBUTES/VARIABLE_VALUEB(variables/157/.ATTRIBUTES/VARIABLE_VALUEB(variables/158/.ATTRIBUTES/VARIABLE_VALUEB(variables/159/.ATTRIBUTES/VARIABLE_VALUEB(variables/160/.ATTRIBUTES/VARIABLE_VALUEB(variables/161/.ATTRIBUTES/VARIABLE_VALUEB(variables/162/.ATTRIBUTES/VARIABLE_VALUEB(variables/163/.ATTRIBUTES/VARIABLE_VALUEB(variables/164/.ATTRIBUTES/VARIABLE_VALUEB(variables/165/.ATTRIBUTES/VARIABLE_VALUEB(variables/166/.ATTRIBUTES/VARIABLE_VALUEB(variables/167/.ATTRIBUTES/VARIABLE_VALUEB(variables/168/.ATTRIBUTES/VARIABLE_VALUEB(variables/169/.ATTRIBUTES/VARIABLE_VALUEB(variables/170/.ATTRIBUTES/VARIABLE_VALUEB(variables/171/.ATTRIBUTES/VARIABLE_VALUEB(variables/172/.ATTRIBUTES/VARIABLE_VALUEB(variables/173/.ATTRIBUTES/VARIABLE_VALUEB(variables/174/.ATTRIBUTES/VARIABLE_VALUEB(variables/175/.ATTRIBUTES/VARIABLE_VALUEB(variables/176/.ATTRIBUTES/VARIABLE_VALUEB(variables/177/.ATTRIBUTES/VARIABLE_VALUEB(variables/178/.ATTRIBUTES/VARIABLE_VALUEB(variables/179/.ATTRIBUTES/VARIABLE_VALUEB(variables/180/.ATTRIBUTES/VARIABLE_VALUEB(variables/181/.ATTRIBUTES/VARIABLE_VALUEB(variables/182/.ATTRIBUTES/VARIABLE_VALUEB(variables/183/.ATTRIBUTES/VARIABLE_VALUEB(variables/184/.ATTRIBUTES/VARIABLE_VALUEB(variables/185/.ATTRIBUTES/VARIABLE_VALUEB(variables/186/.ATTRIBUTES/VARIABLE_VALUEB(variables/187/.ATTRIBUTES/VARIABLE_VALUEB(variables/188/.ATTRIBUTES/VARIABLE_VALUEB(variables/189/.ATTRIBUTES/VARIABLE_VALUEB(variables/190/.ATTRIBUTES/VARIABLE_VALUEB(variables/191/.ATTRIBUTES/VARIABLE_VALUEB(variables/192/.ATTRIBUTES/VARIABLE_VALUEB(variables/193/.ATTRIBUTES/VARIABLE_VALUEB(variables/194/.ATTRIBUTES/VARIABLE_VALUEB(variables/195/.ATTRIBUTES/VARIABLE_VALUEB(variables/196/.ATTRIBUTES/VARIABLE_VALUEB(variables/197/.ATTRIBUTES/VARIABLE_VALUEB(variables/198/.ATTRIBUTES/VARIABLE_VALUE2 RestoreV2/tensor_names£ RestoreV2/shape_and_slicesConst" /device:CPU:0* _output_shapes :Č* dtype0*¦ valueœB™ČBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB2 RestoreV2/shape_and_slicesš RestoreV2 RestoreV2 file_prefixRestoreV2/tensor_names:output:0#RestoreV2/shape_and_slices:output:0" /device:CPU:0*¶ _output_shapes£  ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::*Ł dtypesĪ Ė2Č 2 RestoreV2X IdentityIdentityRestoreV2:tensors:0* T0 * _output_shapes :2 Identity‹ AssignVariableOpAssignVariableOpassignvariableop_variable_1Identity:output:0* _output_shapes * dtype0 2 AssignVariableOp\ Identity_1IdentityRestoreV2:tensors:1* T0* _output_shapes :2 Identity_1£ AssignVariableOp_1AssignVariableOp-assignvariableop_1_word_embeddings_embeddingsIdentity_1:output:0* _output_shapes * dtype02 AssignVariableOp_1\ Identity_2IdentityRestoreV2:tensors:2* T0* _output_shapes :2 Identity_2¦ AssignVariableOp_2AssignVariableOp0assignvariableop_2_position_embedding_embeddingsIdentity_2:output:0* _output_shapes * dtype02 AssignVariableOp_2\ Identity_3IdentityRestoreV2:tensors:3* T0* _output_shapes :2 Identity_3£ AssignVariableOp_3AssignVariableOp-assignvariableop_3_type_embeddings_embeddingsIdentity_3:output:0* _output_shapes * dtype02 AssignVariableOp_3\ Identity_4IdentityRestoreV2:tensors:4* T0* _output_shapes :2 Identity_4¤ AssignVariableOp_4AssignVariableOp.assignvariableop_4_embeddings_layer_norm_gammaIdentity_4:output:0* _output_shapes * dtype02 AssignVariableOp_4\ Identity_5IdentityRestoreV2:tensors:5* T0* _output_shapes :2 Identity_5£ AssignVariableOp_5AssignVariableOp-assignvariableop_5_embeddings_layer_norm_betaIdentity_5:output:0* _output_shapes * dtype02 AssignVariableOp_5\ Identity_6IdentityRestoreV2:tensors:6* T0* _output_shapes :2 Identity_6ø AssignVariableOp_6AssignVariableOpBassignvariableop_6_transformer_layer_0_self_attention_query_kernelIdentity_6:output:0* _output_shapes * dtype02 AssignVariableOp_6\ Identity_7IdentityRestoreV2:tensors:7* T0* _output_shapes :2 Identity_7¶ AssignVariableOp_7AssignVariableOp@assignvariableop_7_transformer_layer_0_self_attention_query_biasIdentity_7:output:0* _output_shapes * dtype02 AssignVariableOp_7\ Identity_8IdentityRestoreV2:tensors:8* T0* _output_shapes :2 Identity_8¶ AssignVariableOp_8AssignVariableOp@assignvariableop_8_transformer_layer_0_self_attention_key_kernelIdentity_8:output:0* _output_shapes * dtype02 AssignVariableOp_8\ Identity_9IdentityRestoreV2:tensors:9* T0* _output_shapes :2 Identity_9“ AssignVariableOp_9AssignVariableOp>assignvariableop_9_transformer_layer_0_self_attention_key_biasIdentity_9:output:0* _output_shapes * dtype02 AssignVariableOp_9_ Identity_10IdentityRestoreV2:tensors:10* T0* _output_shapes :2 Identity_10¼ AssignVariableOp_10AssignVariableOpCassignvariableop_10_transformer_layer_0_self_attention_value_kernelIdentity_10:output:0* _output_shapes * dtype02 AssignVariableOp_10_ Identity_11IdentityRestoreV2:tensors:11* T0* _output_shapes :2 Identity_11ŗ AssignVariableOp_11AssignVariableOpAassignvariableop_11_transformer_layer_0_self_attention_value_biasIdentity_11:output:0* _output_shapes * dtype02 AssignVariableOp_11_ Identity_12IdentityRestoreV2:tensors:12* T0* _output_shapes :2 Identity_12Ē AssignVariableOp_12AssignVariableOpNassignvariableop_12_transformer_layer_0_self_attention_attention_output_kernelIdentity_12:output:0* _output_shapes * dtype02 AssignVariableOp_12_ Identity_13IdentityRestoreV2:tensors:13* T0* _output_shapes :2 Identity_13Å AssignVariableOp_13AssignVariableOpLassignvariableop_13_transformer_layer_0_self_attention_attention_output_biasIdentity_13:output:0* _output_shapes * dtype02 AssignVariableOp_13_ Identity_14IdentityRestoreV2:tensors:14* T0* _output_shapes :2 Identity_14Ą AssignVariableOp_14AssignVariableOpGassignvariableop_14_transformer_layer_0_self_attention_layer_norm_gammaIdentity_14:output:0* _output_shapes * dtype02 AssignVariableOp_14_ Identity_15IdentityRestoreV2:tensors:15* T0* _output_shapes :2 Identity_15æ AssignVariableOp_15AssignVariableOpFassignvariableop_15_transformer_layer_0_self_attention_layer_norm_betaIdentity_15:output:0* _output_shapes * dtype02 AssignVariableOp_15_ Identity_16IdentityRestoreV2:tensors:16* T0* _output_shapes :2 Identity_16“ AssignVariableOp_16AssignVariableOp;assignvariableop_16_transformer_layer_0_intermediate_kernelIdentity_16:output:0* _output_shapes * dtype02 AssignVariableOp_16_ Identity_17IdentityRestoreV2:tensors:17* T0* _output_shapes :2 Identity_17² AssignVariableOp_17AssignVariableOp9assignvariableop_17_transformer_layer_0_intermediate_biasIdentity_17:output:0* _output_shapes * dtype02 AssignVariableOp_17_ Identity_18IdentityRestoreV2:tensors:18* T0* _output_shapes :2 Identity_18® AssignVariableOp_18AssignVariableOp5assignvariableop_18_transformer_layer_0_output_kernelIdentity_18:output:0* _output_shapes * dtype02 AssignVariableOp_18_ Identity_19IdentityRestoreV2:tensors:19* T0* _output_shapes :2 Identity_19¬ AssignVariableOp_19AssignVariableOp3assignvariableop_19_transformer_layer_0_output_biasIdentity_19:output:0* _output_shapes * dtype02 AssignVariableOp_19_ Identity_20IdentityRestoreV2:tensors:20* T0* _output_shapes :2 Identity_20ø AssignVariableOp_20AssignVariableOp?assignvariableop_20_transformer_layer_0_output_layer_norm_gammaIdentity_20:output:0* _output_shapes * dtype02 AssignVariableOp_20_ Identity_21IdentityRestoreV2:tensors:21* T0* _output_shapes :2 Identity_21· AssignVariableOp_21AssignVariableOp>assignvariableop_21_transformer_layer_0_output_layer_norm_betaIdentity_21:output:0* _output_shapes * dtype02 AssignVariableOp_21_ Identity_22IdentityRestoreV2:tensors:22* T0* _output_shapes :2 Identity_22¼ AssignVariableOp_22AssignVariableOpCassignvariableop_22_transformer_layer_1_self_attention_query_kernelIdentity_22:output:0* _output_shapes * dtype02 AssignVariableOp_22_ Identity_23IdentityRestoreV2:tensors:23* T0* _output_shapes :2 Identity_23ŗ AssignVariableOp_23AssignVariableOpAassignvariableop_23_transformer_layer_1_self_attention_query_biasIdentity_23:output:0* _output_shapes * dtype02 AssignVariableOp_23_ Identity_24IdentityRestoreV2:tensors:24* T0* _output_shapes :2 Identity_24ŗ AssignVariableOp_24AssignVariableOpAassignvariableop_24_transformer_layer_1_self_attention_key_kernelIdentity_24:output:0* _output_shapes * dtype02 AssignVariableOp_24_ Identity_25IdentityRestoreV2:tensors:25* T0* _output_shapes :2 Identity_25ø AssignVariableOp_25AssignVariableOp?assignvariableop_25_transformer_layer_1_self_attention_key_biasIdentity_25:output:0* _output_shapes * dtype02 AssignVariableOp_25_ Identity_26IdentityRestoreV2:tensors:26* T0* _output_shapes :2 Identity_26¼ AssignVariableOp_26AssignVariableOpCassignvariableop_26_transformer_layer_1_self_attention_value_kernelIdentity_26:output:0* _output_shapes * dtype02 AssignVariableOp_26_ Identity_27IdentityRestoreV2:tensors:27* T0* _output_shapes :2 Identity_27ŗ AssignVariableOp_27AssignVariableOpAassignvariableop_27_transformer_layer_1_self_attention_value_biasIdentity_27:output:0* _output_shapes * dtype02 AssignVariableOp_27_ Identity_28IdentityRestoreV2:tensors:28* T0* _output_shapes :2 Identity_28Ē AssignVariableOp_28AssignVariableOpNassignvariableop_28_transformer_layer_1_self_attention_attention_output_kernelIdentity_28:output:0* _output_shapes * dtype02 AssignVariableOp_28_ Identity_29IdentityRestoreV2:tensors:29* T0* _output_shapes :2 Identity_29Å AssignVariableOp_29AssignVariableOpLassignvariableop_29_transformer_layer_1_self_attention_attention_output_biasIdentity_29:output:0* _output_shapes * dtype02 AssignVariableOp_29_ Identity_30IdentityRestoreV2:tensors:30* T0* _output_shapes :2 Identity_30Ą AssignVariableOp_30AssignVariableOpGassignvariableop_30_transformer_layer_1_self_attention_layer_norm_gammaIdentity_30:output:0* _output_shapes * dtype02 AssignVariableOp_30_ Identity_31IdentityRestoreV2:tensors:31* T0* _output_shapes :2 Identity_31æ AssignVariableOp_31AssignVariableOpFassignvariableop_31_transformer_layer_1_self_attention_layer_norm_betaIdentity_31:output:0* _output_shapes * dtype02 AssignVariableOp_31_ Identity_32IdentityRestoreV2:tensors:32* T0* _output_shapes :2 Identity_32“ AssignVariableOp_32AssignVariableOp;assignvariableop_32_transformer_layer_1_intermediate_kernelIdentity_32:output:0* _output_shapes * dtype02 AssignVariableOp_32_ Identity_33IdentityRestoreV2:tensors:33* T0* _output_shapes :2 Identity_33² AssignVariableOp_33AssignVariableOp9assignvariableop_33_transformer_layer_1_intermediate_biasIdentity_33:output:0* _output_shapes * dtype02 AssignVariableOp_33_ Identity_34IdentityRestoreV2:tensors:34* T0* _output_shapes :2 Identity_34® AssignVariableOp_34AssignVariableOp5assignvariableop_34_transformer_layer_1_output_kernelIdentity_34:output:0* _output_shapes * dtype02 AssignVariableOp_34_ Identity_35IdentityRestoreV2:tensors:35* T0* _output_shapes :2 Identity_35¬ AssignVariableOp_35AssignVariableOp3assignvariableop_35_transformer_layer_1_output_biasIdentity_35:output:0* _output_shapes * dtype02 AssignVariableOp_35_ Identity_36IdentityRestoreV2:tensors:36* T0* _output_shapes :2 Identity_36ø AssignVariableOp_36AssignVariableOp?assignvariableop_36_transformer_layer_1_output_layer_norm_gammaIdentity_36:output:0* _output_shapes * dtype02 AssignVariableOp_36_ Identity_37IdentityRestoreV2:tensors:37* T0* _output_shapes :2 Identity_37· AssignVariableOp_37AssignVariableOp>assignvariableop_37_transformer_layer_1_output_layer_norm_betaIdentity_37:output:0* _output_shapes * dtype02 AssignVariableOp_37_ Identity_38IdentityRestoreV2:tensors:38* T0* _output_shapes :2 Identity_38¼ AssignVariableOp_38AssignVariableOpCassignvariableop_38_transformer_layer_2_self_attention_query_kernelIdentity_38:output:0* _output_shapes * dtype02 AssignVariableOp_38_ Identity_39IdentityRestoreV2:tensors:39* T0* _output_shapes :2 Identity_39ŗ AssignVariableOp_39AssignVariableOpAassignvariableop_39_transformer_layer_2_self_attention_query_biasIdentity_39:output:0* _output_shapes * dtype02 AssignVariableOp_39_ Identity_40IdentityRestoreV2:tensors:40* T0* _output_shapes :2 Identity_40ŗ AssignVariableOp_40AssignVariableOpAassignvariableop_40_transformer_layer_2_self_attention_key_kernelIdentity_40:output:0* _output_shapes * dtype02 AssignVariableOp_40_ Identity_41IdentityRestoreV2:tensors:41* T0* _output_shapes :2 Identity_41ø AssignVariableOp_41AssignVariableOp?assignvariableop_41_transformer_layer_2_self_attention_key_biasIdentity_41:output:0* _output_shapes * dtype02 AssignVariableOp_41_ Identity_42IdentityRestoreV2:tensors:42* T0* _output_shapes :2 Identity_42¼ AssignVariableOp_42AssignVariableOpCassignvariableop_42_transformer_layer_2_self_attention_value_kernelIdentity_42:output:0* _output_shapes * dtype02 AssignVariableOp_42_ Identity_43IdentityRestoreV2:tensors:43* T0* _output_shapes :2 Identity_43ŗ AssignVariableOp_43AssignVariableOpAassignvariableop_43_transformer_layer_2_self_attention_value_biasIdentity_43:output:0* _output_shapes * dtype02 AssignVariableOp_43_ Identity_44IdentityRestoreV2:tensors:44* T0* _output_shapes :2 Identity_44Ē AssignVariableOp_44AssignVariableOpNassignvariableop_44_transformer_layer_2_self_attention_attention_output_kernelIdentity_44:output:0* _output_shapes * dtype02 AssignVariableOp_44_ Identity_45IdentityRestoreV2:tensors:45* T0* _output_shapes :2 Identity_45Å AssignVariableOp_45AssignVariableOpLassignvariableop_45_transformer_layer_2_self_attention_attention_output_biasIdentity_45:output:0* _output_shapes * dtype02 AssignVariableOp_45_ Identity_46IdentityRestoreV2:tensors:46* T0* _output_shapes :2 Identity_46Ą AssignVariableOp_46AssignVariableOpGassignvariableop_46_transformer_layer_2_self_attention_layer_norm_gammaIdentity_46:output:0* _output_shapes * dtype02 AssignVariableOp_46_ Identity_47IdentityRestoreV2:tensors:47* T0* _output_shapes :2 Identity_47æ AssignVariableOp_47AssignVariableOpFassignvariableop_47_transformer_layer_2_self_attention_layer_norm_betaIdentity_47:output:0* _output_shapes * dtype02 AssignVariableOp_47_ Identity_48IdentityRestoreV2:tensors:48* T0* _output_shapes :2 Identity_48“ AssignVariableOp_48AssignVariableOp;assignvariableop_48_transformer_layer_2_intermediate_kernelIdentity_48:output:0* _output_shapes * dtype02 AssignVariableOp_48_ Identity_49IdentityRestoreV2:tensors:49* T0* _output_shapes :2 Identity_49² AssignVariableOp_49AssignVariableOp9assignvariableop_49_transformer_layer_2_intermediate_biasIdentity_49:output:0* _output_shapes * dtype02 AssignVariableOp_49_ Identity_50IdentityRestoreV2:tensors:50* T0* _output_shapes :2 Identity_50® AssignVariableOp_50AssignVariableOp5assignvariableop_50_transformer_layer_2_output_kernelIdentity_50:output:0* _output_shapes * dtype02 AssignVariableOp_50_ Identity_51IdentityRestoreV2:tensors:51* T0* _output_shapes :2 Identity_51¬ AssignVariableOp_51AssignVariableOp3assignvariableop_51_transformer_layer_2_output_biasIdentity_51:output:0* _output_shapes * dtype02 AssignVariableOp_51_ Identity_52IdentityRestoreV2:tensors:52* T0* _output_shapes :2 Identity_52ø AssignVariableOp_52AssignVariableOp?assignvariableop_52_transformer_layer_2_output_layer_norm_gammaIdentity_52:output:0* _output_shapes * dtype02 AssignVariableOp_52_ Identity_53IdentityRestoreV2:tensors:53* T0* _output_shapes :2 Identity_53· AssignVariableOp_53AssignVariableOp>assignvariableop_53_transformer_layer_2_output_layer_norm_betaIdentity_53:output:0* _output_shapes * dtype02 AssignVariableOp_53_ Identity_54IdentityRestoreV2:tensors:54* T0* _output_shapes :2 Identity_54¼ AssignVariableOp_54AssignVariableOpCassignvariableop_54_transformer_layer_3_self_attention_query_kernelIdentity_54:output:0* _output_shapes * dtype02 AssignVariableOp_54_ Identity_55IdentityRestoreV2:tensors:55* T0* _output_shapes :2 Identity_55ŗ AssignVariableOp_55AssignVariableOpAassignvariableop_55_transformer_layer_3_self_attention_query_biasIdentity_55:output:0* _output_shapes * dtype02 AssignVariableOp_55_ Identity_56IdentityRestoreV2:tensors:56* T0* _output_shapes :2 Identity_56ŗ AssignVariableOp_56AssignVariableOpAassignvariableop_56_transformer_layer_3_self_attention_key_kernelIdentity_56:output:0* _output_shapes * dtype02 AssignVariableOp_56_ Identity_57IdentityRestoreV2:tensors:57* T0* _output_shapes :2 Identity_57ø AssignVariableOp_57AssignVariableOp?assignvariableop_57_transformer_layer_3_self_attention_key_biasIdentity_57:output:0* _output_shapes * dtype02 AssignVariableOp_57_ Identity_58IdentityRestoreV2:tensors:58* T0* _output_shapes :2 Identity_58¼ AssignVariableOp_58AssignVariableOpCassignvariableop_58_transformer_layer_3_self_attention_value_kernelIdentity_58:output:0* _output_shapes * dtype02 AssignVariableOp_58_ Identity_59IdentityRestoreV2:tensors:59* T0* _output_shapes :2 Identity_59ŗ AssignVariableOp_59AssignVariableOpAassignvariableop_59_transformer_layer_3_self_attention_value_biasIdentity_59:output:0* _output_shapes * dtype02 AssignVariableOp_59_ Identity_60IdentityRestoreV2:tensors:60* T0* _output_shapes :2 Identity_60Ē AssignVariableOp_60AssignVariableOpNassignvariableop_60_transformer_layer_3_self_attention_attention_output_kernelIdentity_60:output:0* _output_shapes * dtype02 AssignVariableOp_60_ Identity_61IdentityRestoreV2:tensors:61* T0* _output_shapes :2 Identity_61Å AssignVariableOp_61AssignVariableOpLassignvariableop_61_transformer_layer_3_self_attention_attention_output_biasIdentity_61:output:0* _output_shapes * dtype02 AssignVariableOp_61_ Identity_62IdentityRestoreV2:tensors:62* T0* _output_shapes :2 Identity_62Ą AssignVariableOp_62AssignVariableOpGassignvariableop_62_transformer_layer_3_self_attention_layer_norm_gammaIdentity_62:output:0* _output_shapes * dtype02 AssignVariableOp_62_ Identity_63IdentityRestoreV2:tensors:63* T0* _output_shapes :2 Identity_63æ AssignVariableOp_63AssignVariableOpFassignvariableop_63_transformer_layer_3_self_attention_layer_norm_betaIdentity_63:output:0* _output_shapes * dtype02 AssignVariableOp_63_ Identity_64IdentityRestoreV2:tensors:64* T0* _output_shapes :2 Identity_64“ AssignVariableOp_64AssignVariableOp;assignvariableop_64_transformer_layer_3_intermediate_kernelIdentity_64:output:0* _output_shapes * dtype02 AssignVariableOp_64_ Identity_65IdentityRestoreV2:tensors:65* T0* _output_shapes :2 Identity_65² AssignVariableOp_65AssignVariableOp9assignvariableop_65_transformer_layer_3_intermediate_biasIdentity_65:output:0* _output_shapes * dtype02 AssignVariableOp_65_ Identity_66IdentityRestoreV2:tensors:66* T0* _output_shapes :2 Identity_66® AssignVariableOp_66AssignVariableOp5assignvariableop_66_transformer_layer_3_output_kernelIdentity_66:output:0* _output_shapes * dtype02 AssignVariableOp_66_ Identity_67IdentityRestoreV2:tensors:67* T0* _output_shapes :2 Identity_67¬ AssignVariableOp_67AssignVariableOp3assignvariableop_67_transformer_layer_3_output_biasIdentity_67:output:0* _output_shapes * dtype02 AssignVariableOp_67_ Identity_68IdentityRestoreV2:tensors:68* T0* _output_shapes :2 Identity_68ø AssignVariableOp_68AssignVariableOp?assignvariableop_68_transformer_layer_3_output_layer_norm_gammaIdentity_68:output:0* _output_shapes * dtype02 AssignVariableOp_68_ Identity_69IdentityRestoreV2:tensors:69* T0* _output_shapes :2 Identity_69· AssignVariableOp_69AssignVariableOp>assignvariableop_69_transformer_layer_3_output_layer_norm_betaIdentity_69:output:0* _output_shapes * dtype02 AssignVariableOp_69_ Identity_70IdentityRestoreV2:tensors:70* T0* _output_shapes :2 Identity_70¼ AssignVariableOp_70AssignVariableOpCassignvariableop_70_transformer_layer_4_self_attention_query_kernelIdentity_70:output:0* _output_shapes * dtype02 AssignVariableOp_70_ Identity_71IdentityRestoreV2:tensors:71* T0* _output_shapes :2 Identity_71ŗ AssignVariableOp_71AssignVariableOpAassignvariableop_71_transformer_layer_4_self_attention_query_biasIdentity_71:output:0* _output_shapes * dtype02 AssignVariableOp_71_ Identity_72IdentityRestoreV2:tensors:72* T0* _output_shapes :2 Identity_72ŗ AssignVariableOp_72AssignVariableOpAassignvariableop_72_transformer_layer_4_self_attention_key_kernelIdentity_72:output:0* _output_shapes * dtype02 AssignVariableOp_72_ Identity_73IdentityRestoreV2:tensors:73* T0* _output_shapes :2 Identity_73ø AssignVariableOp_73AssignVariableOp?assignvariableop_73_transformer_layer_4_self_attention_key_biasIdentity_73:output:0* _output_shapes * dtype02 AssignVariableOp_73_ Identity_74IdentityRestoreV2:tensors:74* T0* _output_shapes :2 Identity_74¼ AssignVariableOp_74AssignVariableOpCassignvariableop_74_transformer_layer_4_self_attention_value_kernelIdentity_74:output:0* _output_shapes * dtype02 AssignVariableOp_74_ Identity_75IdentityRestoreV2:tensors:75* T0* _output_shapes :2 Identity_75ŗ AssignVariableOp_75AssignVariableOpAassignvariableop_75_transformer_layer_4_self_attention_value_biasIdentity_75:output:0* _output_shapes * dtype02 AssignVariableOp_75_ Identity_76IdentityRestoreV2:tensors:76* T0* _output_shapes :2 Identity_76Ē AssignVariableOp_76AssignVariableOpNassignvariableop_76_transformer_layer_4_self_attention_attention_output_kernelIdentity_76:output:0* _output_shapes * dtype02 AssignVariableOp_76_ Identity_77IdentityRestoreV2:tensors:77* T0* _output_shapes :2 Identity_77Å AssignVariableOp_77AssignVariableOpLassignvariableop_77_transformer_layer_4_self_attention_attention_output_biasIdentity_77:output:0* _output_shapes * dtype02 AssignVariableOp_77_ Identity_78IdentityRestoreV2:tensors:78* T0* _output_shapes :2 Identity_78Ą AssignVariableOp_78AssignVariableOpGassignvariableop_78_transformer_layer_4_self_attention_layer_norm_gammaIdentity_78:output:0* _output_shapes * dtype02 AssignVariableOp_78_ Identity_79IdentityRestoreV2:tensors:79* T0* _output_shapes :2 Identity_79æ AssignVariableOp_79AssignVariableOpFassignvariableop_79_transformer_layer_4_self_attention_layer_norm_betaIdentity_79:output:0* _output_shapes * dtype02 AssignVariableOp_79_ Identity_80IdentityRestoreV2:tensors:80* T0* _output_shapes :2 Identity_80“ AssignVariableOp_80AssignVariableOp;assignvariableop_80_transformer_layer_4_intermediate_kernelIdentity_80:output:0* _output_shapes * dtype02 AssignVariableOp_80_ Identity_81IdentityRestoreV2:tensors:81* T0* _output_shapes :2 Identity_81² AssignVariableOp_81AssignVariableOp9assignvariableop_81_transformer_layer_4_intermediate_biasIdentity_81:output:0* _output_shapes * dtype02 AssignVariableOp_81_ Identity_82IdentityRestoreV2:tensors:82* T0* _output_shapes :2 Identity_82® AssignVariableOp_82AssignVariableOp5assignvariableop_82_transformer_layer_4_output_kernelIdentity_82:output:0* _output_shapes * dtype02 AssignVariableOp_82_ Identity_83IdentityRestoreV2:tensors:83* T0* _output_shapes :2 Identity_83¬ AssignVariableOp_83AssignVariableOp3assignvariableop_83_transformer_layer_4_output_biasIdentity_83:output:0* _output_shapes * dtype02 AssignVariableOp_83_ Identity_84IdentityRestoreV2:tensors:84* T0* _output_shapes :2 Identity_84ø AssignVariableOp_84AssignVariableOp?assignvariableop_84_transformer_layer_4_output_layer_norm_gammaIdentity_84:output:0* _output_shapes * dtype02 AssignVariableOp_84_ Identity_85IdentityRestoreV2:tensors:85* T0* _output_shapes :2 Identity_85· AssignVariableOp_85AssignVariableOp>assignvariableop_85_transformer_layer_4_output_layer_norm_betaIdentity_85:output:0* _output_shapes * dtype02 AssignVariableOp_85_ Identity_86IdentityRestoreV2:tensors:86* T0* _output_shapes :2 Identity_86¼ AssignVariableOp_86AssignVariableOpCassignvariableop_86_transformer_layer_5_self_attention_query_kernelIdentity_86:output:0* _output_shapes * dtype02 AssignVariableOp_86_ Identity_87IdentityRestoreV2:tensors:87* T0* _output_shapes :2 Identity_87ŗ AssignVariableOp_87AssignVariableOpAassignvariableop_87_transformer_layer_5_self_attention_query_biasIdentity_87:output:0* _output_shapes * dtype02 AssignVariableOp_87_ Identity_88IdentityRestoreV2:tensors:88* T0* _output_shapes :2 Identity_88ŗ AssignVariableOp_88AssignVariableOpAassignvariableop_88_transformer_layer_5_self_attention_key_kernelIdentity_88:output:0* _output_shapes * dtype02 AssignVariableOp_88_ Identity_89IdentityRestoreV2:tensors:89* T0* _output_shapes :2 Identity_89ø AssignVariableOp_89AssignVariableOp?assignvariableop_89_transformer_layer_5_self_attention_key_biasIdentity_89:output:0* _output_shapes * dtype02 AssignVariableOp_89_ Identity_90IdentityRestoreV2:tensors:90* T0* _output_shapes :2 Identity_90¼ AssignVariableOp_90AssignVariableOpCassignvariableop_90_transformer_layer_5_self_attention_value_kernelIdentity_90:output:0* _output_shapes * dtype02 AssignVariableOp_90_ Identity_91IdentityRestoreV2:tensors:91* T0* _output_shapes :2 Identity_91ŗ AssignVariableOp_91AssignVariableOpAassignvariableop_91_transformer_layer_5_self_attention_value_biasIdentity_91:output:0* _output_shapes * dtype02 AssignVariableOp_91_ Identity_92IdentityRestoreV2:tensors:92* T0* _output_shapes :2 Identity_92Ē AssignVariableOp_92AssignVariableOpNassignvariableop_92_transformer_layer_5_self_attention_attention_output_kernelIdentity_92:output:0* _output_shapes * dtype02 AssignVariableOp_92_ Identity_93IdentityRestoreV2:tensors:93* T0* _output_shapes :2 Identity_93Å AssignVariableOp_93AssignVariableOpLassignvariableop_93_transformer_layer_5_self_attention_attention_output_biasIdentity_93:output:0* _output_shapes * dtype02 AssignVariableOp_93_ Identity_94IdentityRestoreV2:tensors:94* T0* _output_shapes :2 Identity_94Ą AssignVariableOp_94AssignVariableOpGassignvariableop_94_transformer_layer_5_self_attention_layer_norm_gammaIdentity_94:output:0* _output_shapes * dtype02 AssignVariableOp_94_ Identity_95IdentityRestoreV2:tensors:95* T0* _output_shapes :2 Identity_95æ AssignVariableOp_95AssignVariableOpFassignvariableop_95_transformer_layer_5_self_attention_layer_norm_betaIdentity_95:output:0* _output_shapes * dtype02 AssignVariableOp_95_ Identity_96IdentityRestoreV2:tensors:96* T0* _output_shapes :2 Identity_96“ AssignVariableOp_96AssignVariableOp;assignvariableop_96_transformer_layer_5_intermediate_kernelIdentity_96:output:0* _output_shapes * dtype02 AssignVariableOp_96_ Identity_97IdentityRestoreV2:tensors:97* T0* _output_shapes :2 Identity_97² AssignVariableOp_97AssignVariableOp9assignvariableop_97_transformer_layer_5_intermediate_biasIdentity_97:output:0* _output_shapes * dtype02 AssignVariableOp_97_ Identity_98IdentityRestoreV2:tensors:98* T0* _output_shapes :2 Identity_98® AssignVariableOp_98AssignVariableOp5assignvariableop_98_transformer_layer_5_output_kernelIdentity_98:output:0* _output_shapes * dtype02 AssignVariableOp_98_ Identity_99IdentityRestoreV2:tensors:99* T0* _output_shapes :2 Identity_99¬ AssignVariableOp_99AssignVariableOp3assignvariableop_99_transformer_layer_5_output_biasIdentity_99:output:0* _output_shapes * dtype02 AssignVariableOp_99b Identity_100IdentityRestoreV2:tensors:100* T0* _output_shapes :2 Identity_100¼ AssignVariableOp_100AssignVariableOp@assignvariableop_100_transformer_layer_5_output_layer_norm_gammaIdentity_100:output:0* _output_shapes * dtype02 AssignVariableOp_100b Identity_101IdentityRestoreV2:tensors:101* T0* _output_shapes :2 Identity_101» AssignVariableOp_101AssignVariableOp?assignvariableop_101_transformer_layer_5_output_layer_norm_betaIdentity_101:output:0* _output_shapes * dtype02 AssignVariableOp_101b Identity_102IdentityRestoreV2:tensors:102* T0* _output_shapes :2 Identity_102Ą AssignVariableOp_102AssignVariableOpDassignvariableop_102_transformer_layer_6_self_attention_query_kernelIdentity_102:output:0* _output_shapes * dtype02 AssignVariableOp_102b Identity_103IdentityRestoreV2:tensors:103* T0* _output_shapes :2 Identity_103¾ AssignVariableOp_103AssignVariableOpBassignvariableop_103_transformer_layer_6_self_attention_query_biasIdentity_103:output:0* _output_shapes * dtype02 AssignVariableOp_103b Identity_104IdentityRestoreV2:tensors:104* T0* _output_shapes :2 Identity_104¾ AssignVariableOp_104AssignVariableOpBassignvariableop_104_transformer_layer_6_self_attention_key_kernelIdentity_104:output:0* _output_shapes * dtype02 AssignVariableOp_104b Identity_105IdentityRestoreV2:tensors:105* T0* _output_shapes :2 Identity_105¼ AssignVariableOp_105AssignVariableOp@assignvariableop_105_transformer_layer_6_self_attention_key_biasIdentity_105:output:0* _output_shapes * dtype02 AssignVariableOp_105b Identity_106IdentityRestoreV2:tensors:106* T0* _output_shapes :2 Identity_106Ą AssignVariableOp_106AssignVariableOpDassignvariableop_106_transformer_layer_6_self_attention_value_kernelIdentity_106:output:0* _output_shapes * dtype02 AssignVariableOp_106b Identity_107IdentityRestoreV2:tensors:107* T0* _output_shapes :2 Identity_107¾ AssignVariableOp_107AssignVariableOpBassignvariableop_107_transformer_layer_6_self_attention_value_biasIdentity_107:output:0* _output_shapes * dtype02 AssignVariableOp_107b Identity_108IdentityRestoreV2:tensors:108* T0* _output_shapes :2 Identity_108Ė AssignVariableOp_108AssignVariableOpOassignvariableop_108_transformer_layer_6_self_attention_attention_output_kernelIdentity_108:output:0* _output_shapes * dtype02 AssignVariableOp_108b Identity_109IdentityRestoreV2:tensors:109* T0* _output_shapes :2 Identity_109É AssignVariableOp_109AssignVariableOpMassignvariableop_109_transformer_layer_6_self_attention_attention_output_biasIdentity_109:output:0* _output_shapes * dtype02 AssignVariableOp_109b Identity_110IdentityRestoreV2:tensors:110* T0* _output_shapes :2 Identity_110Ä AssignVariableOp_110AssignVariableOpHassignvariableop_110_transformer_layer_6_self_attention_layer_norm_gammaIdentity_110:output:0* _output_shapes * dtype02 AssignVariableOp_110b Identity_111IdentityRestoreV2:tensors:111* T0* _output_shapes :2 Identity_111Ć AssignVariableOp_111AssignVariableOpGassignvariableop_111_transformer_layer_6_self_attention_layer_norm_betaIdentity_111:output:0* _output_shapes * dtype02 AssignVariableOp_111b Identity_112IdentityRestoreV2:tensors:112* T0* _output_shapes :2 Identity_112ø AssignVariableOp_112AssignVariableOp  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes : †» £ N__inference_transformer/layer_5_layer_call_and_return_conditional_losses_27186 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_5/ExpandDims/dimõ *self_attention/masked_softmax_5/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_5/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_5/ExpandDims“ %self_attention/masked_softmax_5/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_5/sub/x‚ #self_attention/masked_softmax_5/subSub.self_attention/masked_softmax_5/sub/x:output:03self_attention/masked_softmax_5/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/sub“ %self_attention/masked_softmax_5/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_5/mul/yö #self_attention/masked_softmax_5/mulMul'self_attention/masked_softmax_5/sub:z:0.self_attention/masked_softmax_5/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/mulą #self_attention/masked_softmax_5/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/addŅ 'self_attention/masked_softmax_5/SoftmaxSoftmax'self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_5/Softmax— 'self_attention/dropout_15/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_15/dropout/Const† %self_attention/dropout_15/dropout/MulMul1self_attention/masked_softmax_5/Softmax:softmax:00self_attention/dropout_15/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_15/dropout/Mul³ 'self_attention/dropout_15/dropout/ShapeShape1self_attention/masked_softmax_5/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_15/dropout/Shapeœ >self_attention/dropout_15/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_15/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_15/dropout/random_uniform/RandomUniform© 0self_attention/dropout_15/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_15/dropout/GreaterEqual/yĄ .self_attention/dropout_15/dropout/GreaterEqual GreaterEqualGself_attention/dropout_15/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_15/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_15/dropout/GreaterEqualē &self_attention/dropout_15/dropout/CastCast2self_attention/dropout_15/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_15/dropout/Castü 'self_attention/dropout_15/dropout/Mul_1Mul)self_attention/dropout_15/dropout/Mul:z:0*self_attention/dropout_15/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_15/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_15/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_16/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_16/dropout/ConstĆ dropout_16/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_16/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/dropout/Mul‹ dropout_16/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_16/dropout/Shapeć /dropout_16/dropout/random_uniform/RandomUniform RandomUniform!dropout_16/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_16/dropout/random_uniform/RandomUniform‹ !dropout_16/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_16/dropout/GreaterEqual/yų dropout_16/dropout/GreaterEqual GreaterEqual8dropout_16/dropout/random_uniform/RandomUniform:output:0*dropout_16/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_16/dropout/GreaterEqual® dropout_16/dropout/CastCast#dropout_16/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/dropout/Cast“ dropout_16/dropout/Mul_1Muldropout_16/dropout/Mul:z:0dropout_16/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/dropout/Mul_1{ addAddV2inputs_0dropout_16/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_5/Pow/yž activation_5/PowPowintermediate/add:z:0activation_5/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Powm activation_5/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_5/mul/xž activation_5/mulMulactivation_5/mul/x:output:0activation_5/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul™ activation_5/addAddV2intermediate/add:z:0activation_5/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/addq activation_5/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_5/mul_1/x¤ activation_5/mul_1Mulactivation_5/mul_1/x:output:0activation_5/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_1† activation_5/TanhTanhactivation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Tanhq activation_5/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_5/add_1/x§ activation_5/add_1AddV2activation_5/add_1/x:output:0activation_5/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/add_1q activation_5/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_5/mul_2/x¦ activation_5/mul_2Mulactivation_5/mul_2/x:output:0activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_2 activation_5/mul_3Mulintermediate/add:z:0activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_5/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_17/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_17/dropout/ConstŖ dropout_17/dropout/MulMuloutput/add:z:0!dropout_17/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/dropout/Mulr dropout_17/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_17/dropout/Shapeć /dropout_17/dropout/random_uniform/RandomUniform RandomUniform!dropout_17/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_17/dropout/random_uniform/RandomUniform‹ !dropout_17/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_17/dropout/GreaterEqual/yų dropout_17/dropout/GreaterEqual GreaterEqual8dropout_17/dropout/random_uniform/RandomUniform:output:0*dropout_17/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_17/dropout/GreaterEqual® dropout_17/dropout/CastCast#dropout_17/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/dropout/Cast“ dropout_17/dropout/Mul_1Muldropout_17/dropout/Mul:z:0dropout_17/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/dropout/Mul_1¤ add_1AddV2dropout_17/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ‚™   M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_8072 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¦ ,self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2. ,self_attention/masked_softmax/ExpandDims/dimļ (self_attention/masked_softmax/ExpandDims ExpandDimsinputs_15self_attention/masked_softmax/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax/ExpandDims #self_attention/masked_softmax/sub/xConst* _output_shapes :* dtype0* value B *€?2% #self_attention/masked_softmax/sub/xś !self_attention/masked_softmax/subSub,self_attention/masked_softmax/sub/x:output:01self_attention/masked_softmax/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/sub #self_attention/masked_softmax/mul/yConst* _output_shapes :* dtype0* value B *@Ę2% #self_attention/masked_softmax/mul/yī !self_attention/masked_softmax/mulMul%self_attention/masked_softmax/sub:z:0,self_attention/masked_softmax/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/mulŚ !self_attention/masked_softmax/addAddV2self_attention/Mul:z:0%self_attention/masked_softmax/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/addĢ %self_attention/masked_softmax/SoftmaxSoftmax%self_attention/masked_softmax/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/masked_softmax/SoftmaxĖ self_attention/dropout/IdentityIdentity/self_attention/masked_softmax/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2! self_attention/dropout/Identityż self_attention/einsum_1/EinsumEinsum(self_attention/dropout/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/add dropout_1/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/Identityx addAddV2inputsdropout_1/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addi activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation/Pow/y˜ activation/PowPowintermediate/add:z:0activation/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Powi activation/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation/mul/x– activation/mulMulactivation/mul/x:output:0activation/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul“ activation/addAddV2intermediate/add:z:0activation/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/addm activation/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation/mul_1/xœ activation/mul_1Mulactivation/mul_1/x:output:0activation/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_1€ activation/TanhTanhactivation/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Tanhm activation/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation/add_1/xŸ activation/add_1AddV2activation/add_1/x:output:0activation/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/add_1m activation/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation/mul_2/xž activation/mul_2Mulactivation/mul_2/x:output:0activation/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_2— activation/mul_3Mulintermediate/add:z:0activation/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpŻ output/einsum/EinsumEinsumactivation/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add„ dropout_2/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/Identity£ add_1AddV2dropout_2/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : „ ļ 3__inference_transformer/layer_6_layer_call_fn_27642 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_101482 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : § B &__inference_lambda_layer_call_fn_29258 inputs identity½ PartitionedCallPartitionedCallinputs* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_lambda_layer_call_and_return_conditional_losses_119982 PartitionedCallm IdentityIdentityPartitionedCall:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs žš £ N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_27604 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_6/ExpandDims/dimõ *self_attention/masked_softmax_6/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_6/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_6/ExpandDims“ %self_attention/masked_softmax_6/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_6/sub/x‚ #self_attention/masked_softmax_6/subSub.self_attention/masked_softmax_6/sub/x:output:03self_attention/masked_softmax_6/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/sub“ %self_attention/masked_softmax_6/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_6/mul/yö #self_attention/masked_softmax_6/mulMul'self_attention/masked_softmax_6/sub:z:0.self_attention/masked_softmax_6/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/mulą #self_attention/masked_softmax_6/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/addŅ 'self_attention/masked_softmax_6/SoftmaxSoftmax'self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_6/SoftmaxÓ "self_attention/dropout_18/IdentityIdentity1self_attention/masked_softmax_6/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_18/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_18/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_19/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/Identity{ addAddV2inputs_0dropout_19/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_6/Pow/yž activation_6/PowPowintermediate/add:z:0activation_6/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Powm activation_6/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_6/mul/xž activation_6/mulMulactivation_6/mul/x:output:0activation_6/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul™ activation_6/addAddV2intermediate/add:z:0activation_6/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/addq activation_6/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_6/mul_1/x¤ activation_6/mul_1Mulactivation_6/mul_1/x:output:0activation_6/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_1† activation_6/TanhTanhactivation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Tanhq activation_6/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_6/add_1/x§ activation_6/add_1AddV2activation_6/add_1/x:output:0activation_6/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/add_1q activation_6/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_6/mul_2/x¦ activation_6/mul_2Mulactivation_6/mul_2/x:output:0activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_2 activation_6/mul_3Mulintermediate/add:z:0activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_6/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_20/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/Identity¤ add_1AddV2dropout_20/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Æŗ   M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_8657 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_2/ExpandDims/dimõ *self_attention/masked_softmax_2/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_2/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_2/ExpandDims“ %self_attention/masked_softmax_2/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_2/sub/x‚ #self_attention/masked_softmax_2/subSub.self_attention/masked_softmax_2/sub/x:output:03self_attention/masked_softmax_2/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/sub“ %self_attention/masked_softmax_2/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_2/mul/yö #self_attention/masked_softmax_2/mulMul'self_attention/masked_softmax_2/sub:z:0.self_attention/masked_softmax_2/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/mulą #self_attention/masked_softmax_2/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/addŅ 'self_attention/masked_softmax_2/SoftmaxSoftmax'self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_2/Softmax• &self_attention/dropout_6/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2( &self_attention/dropout_6/dropout/Constƒ $self_attention/dropout_6/dropout/MulMul1self_attention/masked_softmax_2/Softmax:softmax:0/self_attention/dropout_6/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout_6/dropout/Mul± &self_attention/dropout_6/dropout/ShapeShape1self_attention/masked_softmax_2/Softmax:softmax:0* T0* _output_shapes :2( &self_attention/dropout_6/dropout/Shape™ =self_attention/dropout_6/dropout/random_uniform/RandomUniform RandomUniform/self_attention/dropout_6/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02? =self_attention/dropout_6/dropout/random_uniform/RandomUniform§ /self_attention/dropout_6/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=21 /self_attention/dropout_6/dropout/GreaterEqual/y¼ -self_attention/dropout_6/dropout/GreaterEqual GreaterEqualFself_attention/dropout_6/dropout/random_uniform/RandomUniform:output:08self_attention/dropout_6/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2/ -self_attention/dropout_6/dropout/GreaterEqualä %self_attention/dropout_6/dropout/CastCast1self_attention/dropout_6/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_6/dropout/Castų &self_attention/dropout_6/dropout/Mul_1Mul(self_attention/dropout_6/dropout/Mul:z:0)self_attention/dropout_6/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_6/dropout/Mul_1’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_6/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addw dropout_7/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_7/dropout/ConstĄ dropout_7/dropout/MulMul'self_attention/attention_output/add:z:0 dropout_7/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/Mul‰ dropout_7/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_7/dropout/Shapeą .dropout_7/dropout/random_uniform/RandomUniform RandomUniform dropout_7/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_7/dropout/random_uniform/RandomUniform‰ dropout_7/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_7/dropout/GreaterEqual/yō dropout_7/dropout/GreaterEqual GreaterEqual7dropout_7/dropout/random_uniform/RandomUniform:output:0)dropout_7/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/GreaterEqual« dropout_7/dropout/CastCast"dropout_7/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/Cast° dropout_7/dropout/Mul_1Muldropout_7/dropout/Mul:z:0dropout_7/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/Mul_1x addAddV2inputsdropout_7/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_2/Pow/yž activation_2/PowPowintermediate/add:z:0activation_2/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Powm activation_2/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_2/mul/xž activation_2/mulMulactivation_2/mul/x:output:0activation_2/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul™ activation_2/addAddV2intermediate/add:z:0activation_2/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/addq activation_2/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_2/mul_1/x¤ activation_2/mul_1Mulactivation_2/mul_1/x:output:0activation_2/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_1† activation_2/TanhTanhactivation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Tanhq activation_2/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_2/add_1/x§ activation_2/add_1AddV2activation_2/add_1/x:output:0activation_2/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/add_1q activation_2/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_2/mul_2/x¦ activation_2/mul_2Mulactivation_2/mul_2/x:output:0activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_2 activation_2/mul_3Mulintermediate/add:z:0activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_2/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addw dropout_8/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_8/dropout/Const§ dropout_8/dropout/MulMuloutput/add:z:0 dropout_8/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/Mulp dropout_8/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_8/dropout/Shapeą .dropout_8/dropout/random_uniform/RandomUniform RandomUniform dropout_8/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_8/dropout/random_uniform/RandomUniform‰ dropout_8/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_8/dropout/GreaterEqual/yō dropout_8/dropout/GreaterEqual GreaterEqual7dropout_8/dropout/random_uniform/RandomUniform:output:0)dropout_8/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/GreaterEqual« dropout_8/dropout/CastCast"dropout_8/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/Cast° dropout_8/dropout/Mul_1Muldropout_8/dropout/Mul:z:0dropout_8/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/Mul_1£ add_1AddV2dropout_8/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ޹ £ N__inference_transformer/layer_0_layer_call_and_return_conditional_losses_25631 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¦ ,self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2. ,self_attention/masked_softmax/ExpandDims/dimļ (self_attention/masked_softmax/ExpandDims ExpandDimsinputs_15self_attention/masked_softmax/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax/ExpandDims #self_attention/masked_softmax/sub/xConst* _output_shapes :* dtype0* value B *€?2% #self_attention/masked_softmax/sub/xś !self_attention/masked_softmax/subSub,self_attention/masked_softmax/sub/x:output:01self_attention/masked_softmax/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/sub #self_attention/masked_softmax/mul/yConst* _output_shapes :* dtype0* value B *@Ę2% #self_attention/masked_softmax/mul/yī !self_attention/masked_softmax/mulMul%self_attention/masked_softmax/sub:z:0,self_attention/masked_softmax/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/mulŚ !self_attention/masked_softmax/addAddV2self_attention/Mul:z:0%self_attention/masked_softmax/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/addĢ %self_attention/masked_softmax/SoftmaxSoftmax%self_attention/masked_softmax/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/masked_softmax/Softmax‘ $self_attention/dropout/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2& $self_attention/dropout/dropout/Constū "self_attention/dropout/dropout/MulMul/self_attention/masked_softmax/Softmax:softmax:0-self_attention/dropout/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout/dropout/Mul« $self_attention/dropout/dropout/ShapeShape/self_attention/masked_softmax/Softmax:softmax:0* T0* _output_shapes :2& $self_attention/dropout/dropout/Shape“ ;self_attention/dropout/dropout/random_uniform/RandomUniform RandomUniform-self_attention/dropout/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02= ;self_attention/dropout/dropout/random_uniform/RandomUniform£ -self_attention/dropout/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2/ -self_attention/dropout/dropout/GreaterEqual/y“ +self_attention/dropout/dropout/GreaterEqual GreaterEqualDself_attention/dropout/dropout/random_uniform/RandomUniform:output:06self_attention/dropout/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2- +self_attention/dropout/dropout/GreaterEqualŽ #self_attention/dropout/dropout/CastCast/self_attention/dropout/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/dropout/dropout/Castš $self_attention/dropout/dropout/Mul_1Mul&self_attention/dropout/dropout/Mul:z:0'self_attention/dropout/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout/dropout/Mul_1ż self_attention/einsum_1/EinsumEinsum(self_attention/dropout/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addw dropout_1/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_1/dropout/ConstĄ dropout_1/dropout/MulMul'self_attention/attention_output/add:z:0 dropout_1/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/Mul‰ dropout_1/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_1/dropout/Shapeą .dropout_1/dropout/random_uniform/RandomUniform RandomUniform dropout_1/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_1/dropout/random_uniform/RandomUniform‰ dropout_1/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_1/dropout/GreaterEqual/yō dropout_1/dropout/GreaterEqual GreaterEqual7dropout_1/dropout/random_uniform/RandomUniform:output:0)dropout_1/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/GreaterEqual« dropout_1/dropout/CastCast"dropout_1/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/Cast° dropout_1/dropout/Mul_1Muldropout_1/dropout/Mul:z:0dropout_1/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/Mul_1z addAddV2inputs_0dropout_1/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addi activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation/Pow/y˜ activation/PowPowintermediate/add:z:0activation/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Powi activation/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation/mul/x– activation/mulMulactivation/mul/x:output:0activation/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul“ activation/addAddV2intermediate/add:z:0activation/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/addm activation/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation/mul_1/xœ activation/mul_1Mulactivation/mul_1/x:output:0activation/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_1€ activation/TanhTanhactivation/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Tanhm activation/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation/add_1/xŸ activation/add_1AddV2activation/add_1/x:output:0activation/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/add_1m activation/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation/mul_2/xž activation/mul_2Mulactivation/mul_2/x:output:0activation/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_2— activation/mul_3Mulintermediate/add:z:0activation/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpŻ output/einsum/EinsumEinsumactivation/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addw dropout_2/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_2/dropout/Const§ dropout_2/dropout/MulMuloutput/add:z:0 dropout_2/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/Mulp dropout_2/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_2/dropout/Shapeą .dropout_2/dropout/random_uniform/RandomUniform RandomUniform dropout_2/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_2/dropout/random_uniform/RandomUniform‰ dropout_2/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_2/dropout/GreaterEqual/yō dropout_2/dropout/GreaterEqual GreaterEqual7dropout_2/dropout/random_uniform/RandomUniform:output:0)dropout_2/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/GreaterEqual« dropout_2/dropout/CastCast"dropout_2/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/Cast° dropout_2/dropout/Mul_1Muldropout_2/dropout/Mul:z:0dropout_2/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/Mul_1£ add_1AddV2dropout_2/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : žš £ N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_28537 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_9/ExpandDims/dimõ *self_attention/masked_softmax_9/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_9/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_9/ExpandDims“ %self_attention/masked_softmax_9/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_9/sub/x‚ #self_attention/masked_softmax_9/subSub.self_attention/masked_softmax_9/sub/x:output:03self_attention/masked_softmax_9/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/sub“ %self_attention/masked_softmax_9/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_9/mul/yö #self_attention/masked_softmax_9/mulMul'self_attention/masked_softmax_9/sub:z:0.self_attention/masked_softmax_9/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/mulą #self_attention/masked_softmax_9/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/addŅ 'self_attention/masked_softmax_9/SoftmaxSoftmax'self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_9/SoftmaxÓ "self_attention/dropout_27/IdentityIdentity1self_attention/masked_softmax_9/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_27/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_27/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_28/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/Identity{ addAddV2inputs_0dropout_28/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_9/Pow/yž activation_9/PowPowintermediate/add:z:0activation_9/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Powm activation_9/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_9/mul/xž activation_9/mulMulactivation_9/mul/x:output:0activation_9/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul™ activation_9/addAddV2intermediate/add:z:0activation_9/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/addq activation_9/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_9/mul_1/x¤ activation_9/mul_1Mulactivation_9/mul_1/x:output:0activation_9/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_1† activation_9/TanhTanhactivation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Tanhq activation_9/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_9/add_1/x§ activation_9/add_1AddV2activation_9/add_1/x:output:0activation_9/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/add_1q activation_9/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_9/mul_2/x¦ activation_9/mul_2Mulactivation_9/mul_2/x:output:0activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_2 activation_9/mul_3Mulintermediate/add:z:0activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_9/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_29/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/Identity¤ add_1AddV2dropout_29/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : š ” N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_11186 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_9/ExpandDims/dimõ *self_attention/masked_softmax_9/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_9/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_9/ExpandDims“ %self_attention/masked_softmax_9/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_9/sub/x‚ #self_attention/masked_softmax_9/subSub.self_attention/masked_softmax_9/sub/x:output:03self_attention/masked_softmax_9/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/sub“ %self_attention/masked_softmax_9/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_9/mul/yö #self_attention/masked_softmax_9/mulMul'self_attention/masked_softmax_9/sub:z:0.self_attention/masked_softmax_9/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/mulą #self_attention/masked_softmax_9/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/addŅ 'self_attention/masked_softmax_9/SoftmaxSoftmax'self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_9/SoftmaxÓ "self_attention/dropout_27/IdentityIdentity1self_attention/masked_softmax_9/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_27/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_27/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_28/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/Identityy addAddV2inputsdropout_28/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_9/Pow/yž activation_9/PowPowintermediate/add:z:0activation_9/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Powm activation_9/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_9/mul/xž activation_9/mulMulactivation_9/mul/x:output:0activation_9/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul™ activation_9/addAddV2intermediate/add:z:0activation_9/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/addq activation_9/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_9/mul_1/x¤ activation_9/mul_1Mulactivation_9/mul_1/x:output:0activation_9/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_1† activation_9/TanhTanhactivation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Tanhq activation_9/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_9/add_1/x§ activation_9/add_1AddV2activation_9/add_1/x:output:0activation_9/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/add_1q activation_9/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_9/mul_2/x¦ activation_9/mul_2Mulactivation_9/mul_2/x:output:0activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_2 activation_9/mul_3Mulintermediate/add:z:0activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_9/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_29/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/Identity¤ add_1AddV2dropout_29/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Č  I__inference_word_embeddings_layer_call_and_return_conditional_losses_7628 inputs gather_resource identity_1ˆq Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’2 Reshape/shapek ReshapeReshapeinputsReshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2 ReshapeŒ GatherResourceGathergather_resourceReshape:output:0* Tindices0*( _output_shapes : ’’’’’’’’’€* dtype02 Gatherd IdentityIdentityGather:output:0* T0*( _output_shapes : ’’’’’’’’’€2 IdentityD ShapeShapeinputs* T0* _output_shapes :2 Shapem concat/values_1Const* _output_shapes :* dtype0* valueB :€2 concat/values_1\ concat/axisConst* _output_shapes :* dtype0* value B:2 concat/axisŠ concatConcatV2Shape:output:0concat/values_1:output:0concat/axis:output:0* N* T0* _output_shapes :2 concat… Reshape_1ReshapeIdentity:output:0concat:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Reshape_1x Identity_1IdentityReshape_1:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1"! identity_1Identity_1:output:0*3 _input_shapes" : ’’’’’’’’’ ’’’’’’’’’::XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes :  u /__inference_word_embeddings_layer_call_fn_25332 inputs unknown identityˆ¢StatefulPartitionedCall÷ StatefulPartitionedCallStatefulPartitionedCallinputsunknown* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_word_embeddings_layer_call_and_return_conditional_losses_76282 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*3 _input_shapes" : ’’’’’’’’’ ’’’’’’’’’:22 StatefulPartitionedCallStatefulPartitionedCall:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes : † š 4__inference_transformer/layer_10_layer_call_fn_28924 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŅ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_115322 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : š   M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_9456 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_4/ExpandDims/dimõ *self_attention/masked_softmax_4/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_4/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_4/ExpandDims“ %self_attention/masked_softmax_4/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_4/sub/x‚ #self_attention/masked_softmax_4/subSub.self_attention/masked_softmax_4/sub/x:output:03self_attention/masked_softmax_4/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/sub“ %self_attention/masked_softmax_4/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_4/mul/yö #self_attention/masked_softmax_4/mulMul'self_attention/masked_softmax_4/sub:z:0.self_attention/masked_softmax_4/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/mulą #self_attention/masked_softmax_4/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/addŅ 'self_attention/masked_softmax_4/SoftmaxSoftmax'self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_4/SoftmaxÓ "self_attention/dropout_12/IdentityIdentity1self_attention/masked_softmax_4/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_12/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_12/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_13/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/Identityy addAddV2inputsdropout_13/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_4/Pow/yž activation_4/PowPowintermediate/add:z:0activation_4/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Powm activation_4/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_4/mul/xž activation_4/mulMulactivation_4/mul/x:output:0activation_4/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul™ activation_4/addAddV2intermediate/add:z:0activation_4/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/addq activation_4/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_4/mul_1/x¤ activation_4/mul_1Mulactivation_4/mul_1/x:output:0activation_4/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_1† activation_4/TanhTanhactivation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Tanhq activation_4/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_4/add_1/x§ activation_4/add_1AddV2activation_4/add_1/x:output:0activation_4/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/add_1q activation_4/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_4/mul_2/x¦ activation_4/mul_2Mulactivation_4/mul_2/x:output:0activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_2 activation_4/mul_3Mulintermediate/add:z:0activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_4/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_14/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/Identity¤ add_1AddV2dropout_14/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ÷ŗ   M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_9349 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_4/ExpandDims/dimõ *self_attention/masked_softmax_4/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_4/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_4/ExpandDims“ %self_attention/masked_softmax_4/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_4/sub/x‚ #self_attention/masked_softmax_4/subSub.self_attention/masked_softmax_4/sub/x:output:03self_attention/masked_softmax_4/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/sub“ %self_attention/masked_softmax_4/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_4/mul/yö #self_attention/masked_softmax_4/mulMul'self_attention/masked_softmax_4/sub:z:0.self_attention/masked_softmax_4/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/mulą #self_attention/masked_softmax_4/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/addŅ 'self_attention/masked_softmax_4/SoftmaxSoftmax'self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_4/Softmax— 'self_attention/dropout_12/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_12/dropout/Const† %self_attention/dropout_12/dropout/MulMul1self_attention/masked_softmax_4/Softmax:softmax:00self_attention/dropout_12/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_12/dropout/Mul³ 'self_attention/dropout_12/dropout/ShapeShape1self_attention/masked_softmax_4/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_12/dropout/Shapeœ >self_attention/dropout_12/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_12/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_12/dropout/random_uniform/RandomUniform© 0self_attention/dropout_12/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_12/dropout/GreaterEqual/yĄ .self_attention/dropout_12/dropout/GreaterEqual GreaterEqualGself_attention/dropout_12/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_12/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_12/dropout/GreaterEqualē &self_attention/dropout_12/dropout/CastCast2self_attention/dropout_12/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_12/dropout/Castü 'self_attention/dropout_12/dropout/Mul_1Mul)self_attention/dropout_12/dropout/Mul:z:0*self_attention/dropout_12/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_12/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_12/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_13/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_13/dropout/ConstĆ dropout_13/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_13/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/dropout/Mul‹ dropout_13/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_13/dropout/Shapeć /dropout_13/dropout/random_uniform/RandomUniform RandomUniform!dropout_13/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_13/dropout/random_uniform/RandomUniform‹ !dropout_13/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_13/dropout/GreaterEqual/yų dropout_13/dropout/GreaterEqual GreaterEqual8dropout_13/dropout/random_uniform/RandomUniform:output:0*dropout_13/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_13/dropout/GreaterEqual® dropout_13/dropout/CastCast#dropout_13/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/dropout/Cast“ dropout_13/dropout/Mul_1Muldropout_13/dropout/Mul:z:0dropout_13/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/dropout/Mul_1y addAddV2inputsdropout_13/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_4/Pow/yž activation_4/PowPowintermediate/add:z:0activation_4/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Powm activation_4/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_4/mul/xž activation_4/mulMulactivation_4/mul/x:output:0activation_4/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul™ activation_4/addAddV2intermediate/add:z:0activation_4/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/addq activation_4/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_4/mul_1/x¤ activation_4/mul_1Mulactivation_4/mul_1/x:output:0activation_4/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_1† activation_4/TanhTanhactivation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Tanhq activation_4/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_4/add_1/x§ activation_4/add_1AddV2activation_4/add_1/x:output:0activation_4/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/add_1q activation_4/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_4/mul_2/x¦ activation_4/mul_2Mulactivation_4/mul_2/x:output:0activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_2 activation_4/mul_3Mulintermediate/add:z:0activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_4/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_14/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_14/dropout/ConstŖ dropout_14/dropout/MulMuloutput/add:z:0!dropout_14/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/dropout/Mulr dropout_14/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_14/dropout/Shapeć /dropout_14/dropout/random_uniform/RandomUniform RandomUniform!dropout_14/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_14/dropout/random_uniform/RandomUniform‹ !dropout_14/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_14/dropout/GreaterEqual/yų dropout_14/dropout/GreaterEqual GreaterEqual8dropout_14/dropout/random_uniform/RandomUniform:output:0*dropout_14/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_14/dropout/GreaterEqual® dropout_14/dropout/CastCast#dropout_14/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/dropout/Cast“ dropout_14/dropout/Mul_1Muldropout_14/dropout/Mul:z:0dropout_14/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/dropout/Mul_1¤ add_1AddV2dropout_14/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : „ ļ 3__inference_transformer/layer_7_layer_call_fn_27953 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_104942 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ģņ õy N__inference_transformer_encoder_layer_call_and_return_conditional_losses_24497 inputs_0 inputs_1 inputs_2# word_embeddings_gather_resource. *position_embedding_readvariableop_resource2 .type_embeddings_matmul_readvariableop_resource? ;embeddings_layer_norm_batchnorm_mul_readvariableop_resource; 7embeddings_layer_norm_batchnorm_readvariableop_resourceR Ntransformer_layer_0_self_attention_query_einsum_einsum_readvariableop_resourceH Dtransformer_layer_0_self_attention_query_add_readvariableop_resourceP Ltransformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resourceF Btransformer_layer_0_self_attention_key_add_readvariableop_resourceR Ntransformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resourceH Dtransformer_layer_0_self_attention_value_add_readvariableop_resource] Ytransformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resourceS Otransformer_layer_0_self_attention_attention_output_add_readvariableop_resourceW Stransformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceS Otransformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resourceJ Ftransformer_layer_0_intermediate_einsum_einsum_readvariableop_resource@ abde28 6transformer/layer_0/self_attention/query/einsum/Einsum’ ;transformer/layer_0/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_0_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_0/self_attention/query/add/ReadVariableOp® ,transformer/layer_0/self_attention/query/addAddV2?transformer/layer_0/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_0/self_attention/query/addœ Ctransformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpĒ 4transformer/layer_0/self_attention/key/einsum/EinsumEinsumdropout/Identity:output:0Ktransformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_0/self_attention/key/einsum/Einsumł 9transformer/layer_0/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_0_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_0/self_attention/key/add/ReadVariableOp¦ *transformer/layer_0/self_attention/key/addAddV2=transformer/layer_0/self_attention/key/einsum/Einsum:output:0Atransformer/layer_0/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_0/self_attention/key/add¢ Etransformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpĶ 6transformer/layer_0/self_attention/value/einsum/EinsumEinsumdropout/Identity:output:0Mtransformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_0/self_attention/value/einsum/Einsum’ ;transformer/layer_0/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_0_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_0/self_attention/value/add/ReadVariableOp® ,transformer/layer_0/self_attention/value/addAddV2?transformer/layer_0/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_0/self_attention/value/addÄ 0transformer/layer_0/self_attention/einsum/EinsumEinsum.transformer/layer_0/self_attention/key/add:z:00transformer/layer_0/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_0/self_attention/einsum/Einsum™ (transformer/layer_0/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_0/self_attention/Mul/y‘ &transformer/layer_0/self_attention/MulMul9transformer/layer_0/self_attention/einsum/Einsum:output:01transformer/layer_0/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_0/self_attention/MulĪ @transformer/layer_0/self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2B @transformer/layer_0/self_attention/masked_softmax/ExpandDims/dim¾ abcd24 2transformer/layer_0/self_attention/einsum_1/EinsumĆ Ptransformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_0/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_0/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_0/self_attention/attention_output/einsum/Einsum Ftransformer/layer_0/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_0_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_0/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_0/self_attention/attention_output/addAddV2Jtransformer/layer_0/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_0/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_0/self_attention/attention_output/addŁ &transformer/layer_0/dropout_1/IdentityIdentity;transformer/layer_0/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2( &transformer/layer_0/dropout_1/IdentityĒ transformer/layer_0/addAddV2dropout/Identity:output:0/transformer/layer_0/dropout_1/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_0/addę Ltransformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_0/self_attention_layer_norm/moments/meanMeantransformer/layer_0/add:z:0Utransformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_0/self_attention_layer_norm/moments/meanœ Btransformer/layer_0/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_0/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_0/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_0/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_0/add:z:0Ktransformer/layer_0/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_0/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_0/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_0/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_0/self_attention_layer_norm/moments/varianceĆ =transformer/layer_0/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_0/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_0/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_0/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_0/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_0/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_0/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_0/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_0/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_0/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_0/add:z:0?transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_0/self_attention_layer_norm/moments/mean:output:0?transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_0/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_0/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_0/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_0/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_0/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_0/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_0/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_0_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_0/intermediate/einsum/EinsumEinsumAtransformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_0/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_0/intermediate/einsum/Einsumä 3transformer/layer_0/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_0/output/einsum/EinsumŅ -transformer/layer_0/output/add/ReadVariableOpReadVariableOp6transformer_layer_0_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_0/output/add/ReadVariableOpó transformer/layer_0/output/addAddV21transformer/layer_0/output/einsum/Einsum:output:05transformer/layer_0/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_0/output/addĄ &transformer/layer_0/dropout_2/IdentityIdentity"transformer/layer_0/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2( &transformer/layer_0/dropout_2/Identityó transformer/layer_0/add_1AddV2/transformer/layer_0/dropout_2/Identity:output:0Atransformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_0/add_1Ö Dtransformer/layer_0/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_0/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_0/output_layer_norm/moments/meanMeantransformer/layer_0/add_1:z:0Mtransformer/layer_0/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_0/output_layer_norm/moments/mean„ :transformer/layer_0/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_0/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_0/output_layer_norm/moments/StopGradient» ?transformer/layer_0/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_0/add_1:z:0Ctransformer/layer_0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_0/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_0/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_0/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_0/output_layer_norm/moments/varianceMeanCtransformer/layer_0/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_0/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_0/output_layer_norm/moments/variance³ 5transformer/layer_0/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_0/output_layer_norm/batchnorm/add/y³ 3transformer/layer_0/output_layer_norm/batchnorm/addAddV2?transformer/layer_0/output_layer_norm/moments/variance:output:0>transformer/layer_0/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_0/output_layer_norm/batchnorm/addļ 5transformer/layer_0/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_0/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_0/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_0/output_layer_norm/batchnorm/mulMul9transformer/layer_0/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_0/output_layer_norm/batchnorm/mul 5transformer/layer_0/output_layer_norm/batchnorm/mul_1Multransformer/layer_0/add_1:z:07transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_0/output_layer_norm/batchnorm/mul_1« 5transformer/layer_0/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_0/output_layer_norm/moments/mean:output:07transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_0/output_layer_norm/batchnorm/mul_2… >transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_0_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_0/output_layer_norm/batchnorm/subSubFtransformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_0/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_0/output_layer_norm/batchnorm/sub« 5transformer/layer_0/output_layer_norm/batchnorm/add_1AddV29transformer/layer_0/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_0/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_0/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_1/self_attention/query/einsum/EinsumEinsum9transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_1/self_attention/query/einsum/Einsum’ ;transformer/layer_1/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_1_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_1/self_attention/query/add/ReadVariableOp® ,transformer/layer_1/self_attention/query/addAddV2?transformer/layer_1/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_1/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_1/self_attention/query/addœ Ctransformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_1/self_attention/key/einsum/EinsumEinsum9transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_1/self_attention/key/einsum/Einsumł 9transformer/layer_1/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_1_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_1/self_attention/key/add/ReadVariableOp¦ *transformer/layer_1/self_attention/key/addAddV2=transformer/layer_1/self_attention/key/einsum/Einsum:output:0Atransformer/layer_1/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_1/self_attention/key/add¢ Etransformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_1/self_attention/value/einsum/EinsumEinsum9transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_1/self_attention/value/einsum/Einsum’ ;transformer/layer_1/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_1_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_1/self_attention/value/add/ReadVariableOp® ,transformer/layer_1/self_attention/value/addAddV2?transformer/layer_1/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_1/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_1/self_attention/value/addÄ 0transformer/layer_1/self_attention/einsum/EinsumEinsum.transformer/layer_1/self_attention/key/add:z:00transformer/layer_1/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_1/self_attention/einsum/Einsum™ (transformer/layer_1/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_1/self_attention/Mul/y‘ &transformer/layer_1/self_attention/MulMul9transformer/layer_1/self_attention/einsum/Einsum:output:01transformer/layer_1/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_1/self_attention/MulŅ Btransformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dimÄ >transformer/layer_1/self_attention/masked_softmax_1/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_1/self_attention/masked_softmax_1/ExpandDims» 9transformer/layer_1/self_attention/masked_softmax_1/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_1/self_attention/masked_softmax_1/sub/xŅ 7transformer/layer_1/self_attention/masked_softmax_1/subSubBtransformer/layer_1/self_attention/masked_softmax_1/sub/x:output:0Gtransformer/layer_1/self_attention/masked_softmax_1/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_1/self_attention/masked_softmax_1/sub» 9transformer/layer_1/self_attention/masked_softmax_1/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_1/self_attention/masked_softmax_1/mul/yĘ 7transformer/layer_1/self_attention/masked_softmax_1/mulMul;transformer/layer_1/self_attention/masked_softmax_1/sub:z:0Btransformer/layer_1/self_attention/masked_softmax_1/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_1/self_attention/masked_softmax_1/mul° 7transformer/layer_1/self_attention/masked_softmax_1/addAddV2*transformer/layer_1/self_attention/Mul:z:0;transformer/layer_1/self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_1/self_attention/masked_softmax_1/addŽ ;transformer/layer_1/self_attention/masked_softmax_1/SoftmaxSoftmax;transformer/layer_1/self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_1/self_attention/masked_softmax_1/Softmax 5transformer/layer_1/self_attention/dropout_3/IdentityIdentityEtransformer/layer_1/self_attention/masked_softmax_1/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_1/self_attention/dropout_3/IdentityĻ 2transformer/layer_1/self_attention/einsum_1/EinsumEinsum>transformer/layer_1/self_attention/dropout_3/Identity:output:00transformer/layer_1/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_1/self_attention/einsum_1/EinsumĆ Ptransformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_1/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_1/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_1/self_attention/attention_output/einsum/Einsum Ftransformer/layer_1/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_1_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_1/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_1/self_attention/attention_output/addAddV2Jtransformer/layer_1/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_1/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_1/self_attention/attention_output/addŁ &transformer/layer_1/dropout_4/IdentityIdentity;transformer/layer_1/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2( &transformer/layer_1/dropout_4/Identityē transformer/layer_1/addAddV29transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0/transformer/layer_1/dropout_4/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_1/addę Ltransformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_1/self_attention_layer_norm/moments/meanMeantransformer/layer_1/add:z:0Utransformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_1/self_attention_layer_norm/moments/meanœ Btransformer/layer_1/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_1/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_1/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_1/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_1/add:z:0Ktransformer/layer_1/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_1/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_1/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_1/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_1/self_attention_layer_norm/moments/varianceĆ =transformer/layer_1/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_1/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_1/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_1/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_1/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_1/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_1/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_1/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_1/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_1/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_1/add:z:0?transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_1/self_attention_layer_norm/moments/mean:output:0?transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_1/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_1/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_1/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_1/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_1/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_1/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_1/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_1_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_1/intermediate/einsum/EinsumEinsumAtransformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_1/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_1/intermediate/einsum/Einsumä 3transformer/layer_1/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_1/output/einsum/EinsumŅ -transformer/layer_1/output/add/ReadVariableOpReadVariableOp6transformer_layer_1_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_1/output/add/ReadVariableOpó transformer/layer_1/output/addAddV21transformer/layer_1/output/einsum/Einsum:output:05transformer/layer_1/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_1/output/addĄ &transformer/layer_1/dropout_5/IdentityIdentity"transformer/layer_1/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2( &transformer/layer_1/dropout_5/Identityó transformer/layer_1/add_1AddV2/transformer/layer_1/dropout_5/Identity:output:0Atransformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_1/add_1Ö Dtransformer/layer_1/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_1/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_1/output_layer_norm/moments/meanMeantransformer/layer_1/add_1:z:0Mtransformer/layer_1/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_1/output_layer_norm/moments/mean„ :transformer/layer_1/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_1/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_1/output_layer_norm/moments/StopGradient» ?transformer/layer_1/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_1/add_1:z:0Ctransformer/layer_1/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_1/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_1/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_1/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_1/output_layer_norm/moments/varianceMeanCtransformer/layer_1/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_1/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_1/output_layer_norm/moments/variance³ 5transformer/layer_1/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_1/output_layer_norm/batchnorm/add/y³ 3transformer/layer_1/output_layer_norm/batchnorm/addAddV2?transformer/layer_1/output_layer_norm/moments/variance:output:0>transformer/layer_1/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_1/output_layer_norm/batchnorm/addļ 5transformer/layer_1/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_1/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_1/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_1/output_layer_norm/batchnorm/mulMul9transformer/layer_1/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_1/output_layer_norm/batchnorm/mul 5transformer/layer_1/output_layer_norm/batchnorm/mul_1Multransformer/layer_1/add_1:z:07transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_1/output_layer_norm/batchnorm/mul_1« 5transformer/layer_1/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_1/output_layer_norm/moments/mean:output:07transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_1/output_layer_norm/batchnorm/mul_2… >transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_1_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_1/output_layer_norm/batchnorm/subSubFtransformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_1/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_1/output_layer_norm/batchnorm/sub« 5transformer/layer_1/output_layer_norm/batchnorm/add_1AddV29transformer/layer_1/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_1/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_1/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_2/self_attention/query/einsum/EinsumEinsum9transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_2/self_attention/query/einsum/Einsum’ ;transformer/layer_2/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_2_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_2/self_attention/query/add/ReadVariableOp® ,transformer/layer_2/self_attention/query/addAddV2?transformer/layer_2/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_2/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_2/self_attention/query/addœ Ctransformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_2/self_attention/key/einsum/EinsumEinsum9transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_2/self_attention/key/einsum/Einsumł 9transformer/layer_2/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_2_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_2/self_attention/key/add/ReadVariableOp¦ *transformer/layer_2/self_attention/key/addAddV2=transformer/layer_2/self_attention/key/einsum/Einsum:output:0Atransformer/layer_2/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_2/self_attention/key/add¢ Etransformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_2/self_attention/value/einsum/EinsumEinsum9transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_2/self_attention/value/einsum/Einsum’ ;transformer/layer_2/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_2_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_2/self_attention/value/add/ReadVariableOp® ,transformer/layer_2/self_attention/value/addAddV2?transformer/layer_2/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_2/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_2/self_attention/value/addÄ 0transformer/layer_2/self_attention/einsum/EinsumEinsum.transformer/layer_2/self_attention/key/add:z:00transformer/layer_2/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_2/self_attention/einsum/Einsum™ (transformer/layer_2/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_2/self_attention/Mul/y‘ &transformer/layer_2/self_attention/MulMul9transformer/layer_2/self_attention/einsum/Einsum:output:01transformer/layer_2/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_2/self_attention/MulŅ Btransformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dimÄ >transformer/layer_2/self_attention/masked_softmax_2/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_2/self_attention/masked_softmax_2/ExpandDims» 9transformer/layer_2/self_attention/masked_softmax_2/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_2/self_attention/masked_softmax_2/sub/xŅ 7transformer/layer_2/self_attention/masked_softmax_2/subSubBtransformer/layer_2/self_attention/masked_softmax_2/sub/x:output:0Gtransformer/layer_2/self_attention/masked_softmax_2/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_2/self_attention/masked_softmax_2/sub» 9transformer/layer_2/self_attention/masked_softmax_2/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_2/self_attention/masked_softmax_2/mul/yĘ 7transformer/layer_2/self_attention/masked_softmax_2/mulMul;transformer/layer_2/self_attention/masked_softmax_2/sub:z:0Btransformer/layer_2/self_attention/masked_softmax_2/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_2/self_attention/masked_softmax_2/mul° 7transformer/layer_2/self_attention/masked_softmax_2/addAddV2*transformer/layer_2/self_attention/Mul:z:0;transformer/layer_2/self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_2/self_attention/masked_softmax_2/addŽ ;transformer/layer_2/self_attention/masked_softmax_2/SoftmaxSoftmax;transformer/layer_2/self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_2/self_attention/masked_softmax_2/Softmax 5transformer/layer_2/self_attention/dropout_6/IdentityIdentityEtransformer/layer_2/self_attention/masked_softmax_2/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_2/self_attention/dropout_6/IdentityĻ 2transformer/layer_2/self_attention/einsum_1/EinsumEinsum>transformer/layer_2/self_attention/dropout_6/Identity:output:00transformer/layer_2/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_2/self_attention/einsum_1/EinsumĆ Ptransformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_2/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_2/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_2/self_attention/attention_output/einsum/Einsum Ftransformer/layer_2/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_2_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_2/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_2/self_attention/attention_output/addAddV2Jtransformer/layer_2/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_2/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_2/self_attention/attention_output/addŁ &transformer/layer_2/dropout_7/IdentityIdentity;transformer/layer_2/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2( &transformer/layer_2/dropout_7/Identityē transformer/layer_2/addAddV29transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0/transformer/layer_2/dropout_7/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_2/addę Ltransformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_2/self_attention_layer_norm/moments/meanMeantransformer/layer_2/add:z:0Utransformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_2/self_attention_layer_norm/moments/meanœ Btransformer/layer_2/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_2/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_2/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_2/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_2/add:z:0Ktransformer/layer_2/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_2/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_2/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_2/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_2/self_attention_layer_norm/moments/varianceĆ =transformer/layer_2/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_2/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_2/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_2/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_2/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_2/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_2/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_2/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_2/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_2/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_2/add:z:0?transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_2/self_attention_layer_norm/moments/mean:output:0?transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_2/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_2/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_2/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_2/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_2/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_2/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_2/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_2_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_2/intermediate/einsum/EinsumEinsumAtransformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_2/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_2/intermediate/einsum/Einsumä 3transformer/layer_2/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_2/output/einsum/EinsumŅ -transformer/layer_2/output/add/ReadVariableOpReadVariableOp6transformer_layer_2_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_2/output/add/ReadVariableOpó transformer/layer_2/output/addAddV21transformer/layer_2/output/einsum/Einsum:output:05transformer/layer_2/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_2/output/addĄ &transformer/layer_2/dropout_8/IdentityIdentity"transformer/layer_2/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2( &transformer/layer_2/dropout_8/Identityó transformer/layer_2/add_1AddV2/transformer/layer_2/dropout_8/Identity:output:0Atransformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_2/add_1Ö Dtransformer/layer_2/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_2/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_2/output_layer_norm/moments/meanMeantransformer/layer_2/add_1:z:0Mtransformer/layer_2/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_2/output_layer_norm/moments/mean„ :transformer/layer_2/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_2/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_2/output_layer_norm/moments/StopGradient» ?transformer/layer_2/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_2/add_1:z:0Ctransformer/layer_2/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_2/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_2/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_2/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_2/output_layer_norm/moments/varianceMeanCtransformer/layer_2/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_2/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_2/output_layer_norm/moments/variance³ 5transformer/layer_2/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_2/output_layer_norm/batchnorm/add/y³ 3transformer/layer_2/output_layer_norm/batchnorm/addAddV2?transformer/layer_2/output_layer_norm/moments/variance:output:0>transformer/layer_2/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_2/output_layer_norm/batchnorm/addļ 5transformer/layer_2/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_2/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_2/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_2/output_layer_norm/batchnorm/mulMul9transformer/layer_2/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_2/output_layer_norm/batchnorm/mul 5transformer/layer_2/output_layer_norm/batchnorm/mul_1Multransformer/layer_2/add_1:z:07transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_2/output_layer_norm/batchnorm/mul_1« 5transformer/layer_2/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_2/output_layer_norm/moments/mean:output:07transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_2/output_layer_norm/batchnorm/mul_2… >transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_2_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_2/output_layer_norm/batchnorm/subSubFtransformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_2/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_2/output_layer_norm/batchnorm/sub« 5transformer/layer_2/output_layer_norm/batchnorm/add_1AddV29transformer/layer_2/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_2/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_2/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_3/self_attention/query/einsum/EinsumEinsum9transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_3/self_attention/query/einsum/Einsum’ ;transformer/layer_3/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_3_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_3/self_attention/query/add/ReadVariableOp® ,transformer/layer_3/self_attention/query/addAddV2?transformer/layer_3/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_3/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_3/self_attention/query/addœ Ctransformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_3/self_attention/key/einsum/EinsumEinsum9transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_3/self_attention/key/einsum/Einsumł 9transformer/layer_3/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_3_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_3/self_attention/key/add/ReadVariableOp¦ *transformer/layer_3/self_attention/key/addAddV2=transformer/layer_3/self_attention/key/einsum/Einsum:output:0Atransformer/layer_3/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_3/self_attention/key/add¢ Etransformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_3/self_attention/value/einsum/EinsumEinsum9transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_3/self_attention/value/einsum/Einsum’ ;transformer/layer_3/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_3_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_3/self_attention/value/add/ReadVariableOp® ,transformer/layer_3/self_attention/value/addAddV2?transformer/layer_3/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_3/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_3/self_attention/value/addÄ 0transformer/layer_3/self_attention/einsum/EinsumEinsum.transformer/layer_3/self_attention/key/add:z:00transformer/layer_3/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_3/self_attention/einsum/Einsum™ (transformer/layer_3/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_3/self_attention/Mul/y‘ &transformer/layer_3/self_attention/MulMul9transformer/layer_3/self_attention/einsum/Einsum:output:01transformer/layer_3/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_3/self_attention/MulŅ Btransformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dimÄ >transformer/layer_3/self_attention/masked_softmax_3/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_3/self_attention/masked_softmax_3/ExpandDims» 9transformer/layer_3/self_attention/masked_softmax_3/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_3/self_attention/masked_softmax_3/sub/xŅ 7transformer/layer_3/self_attention/masked_softmax_3/subSubBtransformer/layer_3/self_attention/masked_softmax_3/sub/x:output:0Gtransformer/layer_3/self_attention/masked_softmax_3/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_3/self_attention/masked_softmax_3/sub» 9transformer/layer_3/self_attention/masked_softmax_3/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_3/self_attention/masked_softmax_3/mul/yĘ 7transformer/layer_3/self_attention/masked_softmax_3/mulMul;transformer/layer_3/self_attention/masked_softmax_3/sub:z:0Btransformer/layer_3/self_attention/masked_softmax_3/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_3/self_attention/masked_softmax_3/mul° 7transformer/layer_3/self_attention/masked_softmax_3/addAddV2*transformer/layer_3/self_attention/Mul:z:0;transformer/layer_3/self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_3/self_attention/masked_softmax_3/addŽ ;transformer/layer_3/self_attention/masked_softmax_3/SoftmaxSoftmax;transformer/layer_3/self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_3/self_attention/masked_softmax_3/Softmax 5transformer/layer_3/self_attention/dropout_9/IdentityIdentityEtransformer/layer_3/self_attention/masked_softmax_3/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_3/self_attention/dropout_9/IdentityĻ 2transformer/layer_3/self_attention/einsum_1/EinsumEinsum>transformer/layer_3/self_attention/dropout_9/Identity:output:00transformer/layer_3/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_3/self_attention/einsum_1/EinsumĆ Ptransformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_3/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_3/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_3/self_attention/attention_output/einsum/Einsum Ftransformer/layer_3/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_3_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_3/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_3/self_attention/attention_output/addAddV2Jtransformer/layer_3/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_3/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_3/self_attention/attention_output/addŪ 'transformer/layer_3/dropout_10/IdentityIdentity;transformer/layer_3/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_3/dropout_10/Identityč transformer/layer_3/addAddV29transformer/layer_2/output_layer_norm/batchnorm/add_1:z:00transformer/layer_3/dropout_10/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_3/addę Ltransformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_3/self_attention_layer_norm/moments/meanMeantransformer/layer_3/add:z:0Utransformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_3/self_attention_layer_norm/moments/meanœ Btransformer/layer_3/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_3/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_3/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_3/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_3/add:z:0Ktransformer/layer_3/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_3/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_3/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_3/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_3/self_attention_layer_norm/moments/varianceĆ =transformer/layer_3/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_3/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_3/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_3/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_3/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_3/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_3/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_3/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_3/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_3/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_3/add:z:0?transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_3/self_attention_layer_norm/moments/mean:output:0?transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_3/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_3/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_3/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_3/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_3/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_3/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_3/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_3_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_3/intermediate/einsum/EinsumEinsumAtransformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_3/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_3/intermediate/einsum/Einsumä 3transformer/layer_3/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_3/output/einsum/EinsumŅ -transformer/layer_3/output/add/ReadVariableOpReadVariableOp6transformer_layer_3_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_3/output/add/ReadVariableOpó transformer/layer_3/output/addAddV21transformer/layer_3/output/einsum/Einsum:output:05transformer/layer_3/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_3/output/addĀ 'transformer/layer_3/dropout_11/IdentityIdentity"transformer/layer_3/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_3/dropout_11/Identityō transformer/layer_3/add_1AddV20transformer/layer_3/dropout_11/Identity:output:0Atransformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_3/add_1Ö Dtransformer/layer_3/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_3/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_3/output_layer_norm/moments/meanMeantransformer/layer_3/add_1:z:0Mtransformer/layer_3/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_3/output_layer_norm/moments/mean„ :transformer/layer_3/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_3/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_3/output_layer_norm/moments/StopGradient» ?transformer/layer_3/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_3/add_1:z:0Ctransformer/layer_3/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_3/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_3/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_3/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_3/output_layer_norm/moments/varianceMeanCtransformer/layer_3/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_3/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_3/output_layer_norm/moments/variance³ 5transformer/layer_3/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_3/output_layer_norm/batchnorm/add/y³ 3transformer/layer_3/output_layer_norm/batchnorm/addAddV2?transformer/layer_3/output_layer_norm/moments/variance:output:0>transformer/layer_3/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_3/output_layer_norm/batchnorm/addļ 5transformer/layer_3/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_3/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_3/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_3/output_layer_norm/batchnorm/mulMul9transformer/layer_3/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_3/output_layer_norm/batchnorm/mul 5transformer/layer_3/output_layer_norm/batchnorm/mul_1Multransformer/layer_3/add_1:z:07transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_3/output_layer_norm/batchnorm/mul_1« 5transformer/layer_3/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_3/output_layer_norm/moments/mean:output:07transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_3/output_layer_norm/batchnorm/mul_2… >transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_3_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_3/output_layer_norm/batchnorm/subSubFtransformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_3/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_3/output_layer_norm/batchnorm/sub« 5transformer/layer_3/output_layer_norm/batchnorm/add_1AddV29transformer/layer_3/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_3/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_3/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_4/self_attention/query/einsum/EinsumEinsum9transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_4/self_attention/query/einsum/Einsum’ ;transformer/layer_4/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_4_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_4/self_attention/query/add/ReadVariableOp® ,transformer/layer_4/self_attention/query/addAddV2?transformer/layer_4/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_4/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_4/self_attention/query/addœ Ctransformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_4/self_attention/key/einsum/EinsumEinsum9transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_4/self_attention/key/einsum/Einsumł 9transformer/layer_4/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_4_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_4/self_attention/key/add/ReadVariableOp¦ *transformer/layer_4/self_attention/key/addAddV2=transformer/layer_4/self_attention/key/einsum/Einsum:output:0Atransformer/layer_4/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_4/self_attention/key/add¢ Etransformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_4/self_attention/value/einsum/EinsumEinsum9transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_4/self_attention/value/einsum/Einsum’ ;transformer/layer_4/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_4_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_4/self_attention/value/add/ReadVariableOp® ,transformer/layer_4/self_attention/value/addAddV2?transformer/layer_4/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_4/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_4/self_attention/value/addÄ 0transformer/layer_4/self_attention/einsum/EinsumEinsum.transformer/layer_4/self_attention/key/add:z:00transformer/layer_4/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_4/self_attention/einsum/Einsum™ (transformer/layer_4/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_4/self_attention/Mul/y‘ &transformer/layer_4/self_attention/MulMul9transformer/layer_4/self_attention/einsum/Einsum:output:01transformer/layer_4/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_4/self_attention/MulŅ Btransformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dimÄ >transformer/layer_4/self_attention/masked_softmax_4/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_4/self_attention/masked_softmax_4/ExpandDims» 9transformer/layer_4/self_attention/masked_softmax_4/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_4/self_attention/masked_softmax_4/sub/xŅ 7transformer/layer_4/self_attention/masked_softmax_4/subSubBtransformer/layer_4/self_attention/masked_softmax_4/sub/x:output:0Gtransformer/layer_4/self_attention/masked_softmax_4/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_4/self_attention/masked_softmax_4/sub» 9transformer/layer_4/self_attention/masked_softmax_4/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_4/self_attention/masked_softmax_4/mul/yĘ 7transformer/layer_4/self_attention/masked_softmax_4/mulMul;transformer/layer_4/self_attention/masked_softmax_4/sub:z:0Btransformer/layer_4/self_attention/masked_softmax_4/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_4/self_attention/masked_softmax_4/mul° 7transformer/layer_4/self_attention/masked_softmax_4/addAddV2*transformer/layer_4/self_attention/Mul:z:0;transformer/layer_4/self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_4/self_attention/masked_softmax_4/addŽ ;transformer/layer_4/self_attention/masked_softmax_4/SoftmaxSoftmax;transformer/layer_4/self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_4/self_attention/masked_softmax_4/Softmax 6transformer/layer_4/self_attention/dropout_12/IdentityIdentityEtransformer/layer_4/self_attention/masked_softmax_4/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’28 6transformer/layer_4/self_attention/dropout_12/IdentityŠ 2transformer/layer_4/self_attention/einsum_1/EinsumEinsum?transformer/layer_4/self_attention/dropout_12/Identity:output:00transformer/layer_4/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_4/self_attention/einsum_1/EinsumĆ Ptransformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_4/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_4/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_4/self_attention/attention_output/einsum/Einsum Ftransformer/layer_4/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_4_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_4/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_4/self_attention/attention_output/addAddV2Jtransformer/layer_4/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_4/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_4/self_attention/attention_output/addŪ 'transformer/layer_4/dropout_13/IdentityIdentity;transformer/layer_4/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_4/dropout_13/Identityč transformer/layer_4/addAddV29transformer/layer_3/output_layer_norm/batchnorm/add_1:z:00transformer/layer_4/dropout_13/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_4/addę Ltransformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_4/self_attention_layer_norm/moments/meanMeantransformer/layer_4/add:z:0Utransformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_4/self_attention_layer_norm/moments/meanœ Btransformer/layer_4/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_4/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_4/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_4/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_4/add:z:0Ktransformer/layer_4/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_4/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_4/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_4/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_4/self_attention_layer_norm/moments/varianceĆ =transformer/layer_4/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_4/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_4/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_4/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_4/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_4/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_4/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_4/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_4/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_4/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_4/add:z:0?transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_4/self_attention_layer_norm/moments/mean:output:0?transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_4/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_4/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_4/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_4/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_4/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_4/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_4/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_4_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_4/intermediate/einsum/EinsumEinsumAtransformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_4/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_4/intermediate/einsum/Einsumä 3transformer/layer_4/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_4/output/einsum/EinsumŅ -transformer/layer_4/output/add/ReadVariableOpReadVariableOp6transformer_layer_4_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_4/output/add/ReadVariableOpó transformer/layer_4/output/addAddV21transformer/layer_4/output/einsum/Einsum:output:05transformer/layer_4/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_4/output/addĀ 'transformer/layer_4/dropout_14/IdentityIdentity"transformer/layer_4/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_4/dropout_14/Identityō transformer/layer_4/add_1AddV20transformer/layer_4/dropout_14/Identity:output:0Atransformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_4/add_1Ö Dtransformer/layer_4/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_4/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_4/output_layer_norm/moments/meanMeantransformer/layer_4/add_1:z:0Mtransformer/layer_4/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_4/output_layer_norm/moments/mean„ :transformer/layer_4/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_4/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_4/output_layer_norm/moments/StopGradient» ?transformer/layer_4/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_4/add_1:z:0Ctransformer/layer_4/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_4/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_4/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_4/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_4/output_layer_norm/moments/varianceMeanCtransformer/layer_4/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_4/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_4/output_layer_norm/moments/variance³ 5transformer/layer_4/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_4/output_layer_norm/batchnorm/add/y³ 3transformer/layer_4/output_layer_norm/batchnorm/addAddV2?transformer/layer_4/output_layer_norm/moments/variance:output:0>transformer/layer_4/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_4/output_layer_norm/batchnorm/addļ 5transformer/layer_4/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_4/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_4/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_4/output_layer_norm/batchnorm/mulMul9transformer/layer_4/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_4/output_layer_norm/batchnorm/mul 5transformer/layer_4/output_layer_norm/batchnorm/mul_1Multransformer/layer_4/add_1:z:07transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_4/output_layer_norm/batchnorm/mul_1« 5transformer/layer_4/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_4/output_layer_norm/moments/mean:output:07transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_4/output_layer_norm/batchnorm/mul_2… >transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_4_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_4/output_layer_norm/batchnorm/subSubFtransformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_4/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_4/output_layer_norm/batchnorm/sub« 5transformer/layer_4/output_layer_norm/batchnorm/add_1AddV29transformer/layer_4/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_4/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_4/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_5/self_attention/query/einsum/EinsumEinsum9transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_5/self_attention/query/einsum/Einsum’ ;transformer/layer_5/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_5_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_5/self_attention/query/add/ReadVariableOp® ,transformer/layer_5/self_attention/query/addAddV2?transformer/layer_5/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_5/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_5/self_attention/query/addœ Ctransformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_5/self_attention/key/einsum/EinsumEinsum9transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_5/self_attention/key/einsum/Einsumł 9transformer/layer_5/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_5_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_5/self_attention/key/add/ReadVariableOp¦ *transformer/layer_5/self_attention/key/addAddV2=transformer/layer_5/self_attention/key/einsum/Einsum:output:0Atransformer/layer_5/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_5/self_attention/key/add¢ Etransformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_5/self_attention/value/einsum/EinsumEinsum9transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_5/self_attention/value/einsum/Einsum’ ;transformer/layer_5/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_5_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_5/self_attention/value/add/ReadVariableOp® ,transformer/layer_5/self_attention/value/addAddV2?transformer/layer_5/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_5/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_5/self_attention/value/addÄ 0transformer/layer_5/self_attention/einsum/EinsumEinsum.transformer/layer_5/self_attention/key/add:z:00transformer/layer_5/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_5/self_attention/einsum/Einsum™ (transformer/layer_5/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_5/self_attention/Mul/y‘ &transformer/layer_5/self_attention/MulMul9transformer/layer_5/self_attention/einsum/Einsum:output:01transformer/layer_5/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_5/self_attention/MulŅ Btransformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dimÄ >transformer/layer_5/self_attention/masked_softmax_5/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_5/self_attention/masked_softmax_5/ExpandDims» 9transformer/layer_5/self_attention/masked_softmax_5/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_5/self_attention/masked_softmax_5/sub/xŅ 7transformer/layer_5/self_attention/masked_softmax_5/subSubBtransformer/layer_5/self_attention/masked_softmax_5/sub/x:output:0Gtransformer/layer_5/self_attention/masked_softmax_5/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_5/self_attention/masked_softmax_5/sub» 9transformer/layer_5/self_attention/masked_softmax_5/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_5/self_attention/masked_softmax_5/mul/yĘ 7transformer/layer_5/self_attention/masked_softmax_5/mulMul;transformer/layer_5/self_attention/masked_softmax_5/sub:z:0Btransformer/layer_5/self_attention/masked_softmax_5/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_5/self_attention/masked_softmax_5/mul° 7transformer/layer_5/self_attention/masked_softmax_5/addAddV2*transformer/layer_5/self_attention/Mul:z:0;transformer/layer_5/self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_5/self_attention/masked_softmax_5/addŽ ;transformer/layer_5/self_attention/masked_softmax_5/SoftmaxSoftmax;transformer/layer_5/self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_5/self_attention/masked_softmax_5/Softmax 6transformer/layer_5/self_attention/dropout_15/IdentityIdentityEtransformer/layer_5/self_attention/masked_softmax_5/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’28 6transformer/layer_5/self_attention/dropout_15/IdentityŠ 2transformer/layer_5/self_attention/einsum_1/EinsumEinsum?transformer/layer_5/self_attention/dropout_15/Identity:output:00transformer/layer_5/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_5/self_attention/einsum_1/EinsumĆ Ptransformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_5/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_5/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_5/self_attention/attention_output/einsum/Einsum Ftransformer/layer_5/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_5_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_5/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_5/self_attention/attention_output/addAddV2Jtransformer/layer_5/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_5/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_5/self_attention/attention_output/addŪ 'transformer/layer_5/dropout_16/IdentityIdentity;transformer/layer_5/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_5/dropout_16/Identityč transformer/layer_5/addAddV29transformer/layer_4/output_layer_norm/batchnorm/add_1:z:00transformer/layer_5/dropout_16/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_5/addę Ltransformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_5/self_attention_layer_norm/moments/meanMeantransformer/layer_5/add:z:0Utransformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_5/self_attention_layer_norm/moments/meanœ Btransformer/layer_5/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_5/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_5/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_5/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_5/add:z:0Ktransformer/layer_5/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_5/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_5/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_5/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_5/self_attention_layer_norm/moments/varianceĆ =transformer/layer_5/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_5/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_5/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_5/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_5/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_5/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_5/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_5/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_5/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_5/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_5/add:z:0?transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_5/self_attention_layer_norm/moments/mean:output:0?transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_5/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_5/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_5/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_5/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_5/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_5/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_5/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_5_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_5/intermediate/einsum/EinsumEinsumAtransformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_5/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_5/intermediate/einsum/Einsumä 3transformer/layer_5/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_5/output/einsum/EinsumŅ -transformer/layer_5/output/add/ReadVariableOpReadVariableOp6transformer_layer_5_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_5/output/add/ReadVariableOpó transformer/layer_5/output/addAddV21transformer/layer_5/output/einsum/Einsum:output:05transformer/layer_5/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_5/output/addĀ 'transformer/layer_5/dropout_17/IdentityIdentity"transformer/layer_5/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_5/dropout_17/Identityō transformer/layer_5/add_1AddV20transformer/layer_5/dropout_17/Identity:output:0Atransformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_5/add_1Ö Dtransformer/layer_5/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_5/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_5/output_layer_norm/moments/meanMeantransformer/layer_5/add_1:z:0Mtransformer/layer_5/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_5/output_layer_norm/moments/mean„ :transformer/layer_5/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_5/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_5/output_layer_norm/moments/StopGradient» ?transformer/layer_5/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_5/add_1:z:0Ctransformer/layer_5/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_5/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_5/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_5/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_5/output_layer_norm/moments/varianceMeanCtransformer/layer_5/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_5/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_5/output_layer_norm/moments/variance³ 5transformer/layer_5/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_5/output_layer_norm/batchnorm/add/y³ 3transformer/layer_5/output_layer_norm/batchnorm/addAddV2?transformer/layer_5/output_layer_norm/moments/variance:output:0>transformer/layer_5/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_5/output_layer_norm/batchnorm/addļ 5transformer/layer_5/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_5/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_5/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_5/output_layer_norm/batchnorm/mulMul9transformer/layer_5/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_5/output_layer_norm/batchnorm/mul 5transformer/layer_5/output_layer_norm/batchnorm/mul_1Multransformer/layer_5/add_1:z:07transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_5/output_layer_norm/batchnorm/mul_1« 5transformer/layer_5/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_5/output_layer_norm/moments/mean:output:07transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_5/output_layer_norm/batchnorm/mul_2… >transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_5_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_5/output_layer_norm/batchnorm/subSubFtransformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_5/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_5/output_layer_norm/batchnorm/sub« 5transformer/layer_5/output_layer_norm/batchnorm/add_1AddV29transformer/layer_5/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_5/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_5/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_6/self_attention/query/einsum/EinsumEinsum9transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_6/self_attention/query/einsum/Einsum’ ;transformer/layer_6/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_6_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_6/self_attention/query/add/ReadVariableOp® ,transformer/layer_6/self_attention/query/addAddV2?transformer/layer_6/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_6/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_6/self_attention/query/addœ Ctransformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_6/self_attention/key/einsum/EinsumEinsum9transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_6/self_attention/key/einsum/Einsumł 9transformer/layer_6/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_6_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_6/self_attention/key/add/ReadVariableOp¦ *transformer/layer_6/self_attention/key/addAddV2=transformer/layer_6/self_attention/key/einsum/Einsum:output:0Atransformer/layer_6/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_6/self_attention/key/add¢ Etransformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_6/self_attention/value/einsum/EinsumEinsum9transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_6/self_attention/value/einsum/Einsum’ ;transformer/layer_6/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_6_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_6/self_attention/value/add/ReadVariableOp® ,transformer/layer_6/self_attention/value/addAddV2?transformer/layer_6/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_6/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_6/self_attention/value/addÄ 0transformer/layer_6/self_attention/einsum/EinsumEinsum.transformer/layer_6/self_attention/key/add:z:00transformer/layer_6/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_6/self_attention/einsum/Einsum™ (transformer/layer_6/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_6/self_attention/Mul/y‘ &transformer/layer_6/self_attention/MulMul9transformer/layer_6/self_attention/einsum/Einsum:output:01transformer/layer_6/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_6/self_attention/MulŅ Btransformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dimÄ >transformer/layer_6/self_attention/masked_softmax_6/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_6/self_attention/masked_softmax_6/ExpandDims» 9transformer/layer_6/self_attention/masked_softmax_6/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_6/self_attention/masked_softmax_6/sub/xŅ 7transformer/layer_6/self_attention/masked_softmax_6/subSubBtransformer/layer_6/self_attention/masked_softmax_6/sub/x:output:0Gtransformer/layer_6/self_attention/masked_softmax_6/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_6/self_attention/masked_softmax_6/sub» 9transformer/layer_6/self_attention/masked_softmax_6/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_6/self_attention/masked_softmax_6/mul/yĘ 7transformer/layer_6/self_attention/masked_softmax_6/mulMul;transformer/layer_6/self_attention/masked_softmax_6/sub:z:0Btransformer/layer_6/self_attention/masked_softmax_6/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_6/self_attention/masked_softmax_6/mul° 7transformer/layer_6/self_attention/masked_softmax_6/addAddV2*transformer/layer_6/self_attention/Mul:z:0;transformer/layer_6/self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_6/self_attention/masked_softmax_6/addŽ ;transformer/layer_6/self_attention/masked_softmax_6/SoftmaxSoftmax;transformer/layer_6/self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_6/self_attention/masked_softmax_6/Softmax 6transformer/layer_6/self_attention/dropout_18/IdentityIdentityEtransformer/layer_6/self_attention/masked_softmax_6/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’28 6transformer/layer_6/self_attention/dropout_18/IdentityŠ 2transformer/layer_6/self_attention/einsum_1/EinsumEinsum?transformer/layer_6/self_attention/dropout_18/Identity:output:00transformer/layer_6/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_6/self_attention/einsum_1/EinsumĆ Ptransformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_6/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_6/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_6/self_attention/attention_output/einsum/Einsum Ftransformer/layer_6/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_6_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_6/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_6/self_attention/attention_output/addAddV2Jtransformer/layer_6/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_6/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_6/self_attention/attention_output/addŪ 'transformer/layer_6/dropout_19/IdentityIdentity;transformer/layer_6/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_6/dropout_19/Identityč transformer/layer_6/addAddV29transformer/layer_5/output_layer_norm/batchnorm/add_1:z:00transformer/layer_6/dropout_19/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_6/addę Ltransformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_6/self_attention_layer_norm/moments/meanMeantransformer/layer_6/add:z:0Utransformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_6/self_attention_layer_norm/moments/meanœ Btransformer/layer_6/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_6/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_6/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_6/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_6/add:z:0Ktransformer/layer_6/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_6/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_6/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_6/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_6/self_attention_layer_norm/moments/varianceĆ =transformer/layer_6/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_6/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_6/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_6/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_6/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_6/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_6/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_6/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_6/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_6/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_6/add:z:0?transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_6/self_attention_layer_norm/moments/mean:output:0?transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_6/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_6/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_6/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_6/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_6/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_6/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_6/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_6_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_6/intermediate/einsum/EinsumEinsumAtransformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_6/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_6/intermediate/einsum/Einsumä 3transformer/layer_6/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_6/output/einsum/EinsumŅ -transformer/layer_6/output/add/ReadVariableOpReadVariableOp6transformer_layer_6_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_6/output/add/ReadVariableOpó transformer/layer_6/output/addAddV21transformer/layer_6/output/einsum/Einsum:output:05transformer/layer_6/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_6/output/addĀ 'transformer/layer_6/dropout_20/IdentityIdentity"transformer/layer_6/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_6/dropout_20/Identityō transformer/layer_6/add_1AddV20transformer/layer_6/dropout_20/Identity:output:0Atransformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_6/add_1Ö Dtransformer/layer_6/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_6/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_6/output_layer_norm/moments/meanMeantransformer/layer_6/add_1:z:0Mtransformer/layer_6/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_6/output_layer_norm/moments/mean„ :transformer/layer_6/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_6/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_6/output_layer_norm/moments/StopGradient» ?transformer/layer_6/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_6/add_1:z:0Ctransformer/layer_6/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_6/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_6/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_6/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_6/output_layer_norm/moments/varianceMeanCtransformer/layer_6/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_6/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_6/output_layer_norm/moments/variance³ 5transformer/layer_6/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_6/output_layer_norm/batchnorm/add/y³ 3transformer/layer_6/output_layer_norm/batchnorm/addAddV2?transformer/layer_6/output_layer_norm/moments/variance:output:0>transformer/layer_6/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_6/output_layer_norm/batchnorm/addļ 5transformer/layer_6/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_6/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_6/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_6/output_layer_norm/batchnorm/mulMul9transformer/layer_6/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_6/output_layer_norm/batchnorm/mul 5transformer/layer_6/output_layer_norm/batchnorm/mul_1Multransformer/layer_6/add_1:z:07transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_6/output_layer_norm/batchnorm/mul_1« 5transformer/layer_6/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_6/output_layer_norm/moments/mean:output:07transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_6/output_layer_norm/batchnorm/mul_2… >transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_6_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_6/output_layer_norm/batchnorm/subSubFtransformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_6/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_6/output_layer_norm/batchnorm/sub« 5transformer/layer_6/output_layer_norm/batchnorm/add_1AddV29transformer/layer_6/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_6/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_6/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_7/self_attention/query/einsum/EinsumEinsum9transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_7/self_attention/query/einsum/Einsum’ ;transformer/layer_7/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_7_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_7/self_attention/query/add/ReadVariableOp® ,transformer/layer_7/self_attention/query/addAddV2?transformer/layer_7/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_7/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_7/self_attention/query/addœ Ctransformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_7/self_attention/key/einsum/EinsumEinsum9transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_7/self_attention/key/einsum/Einsumł 9transformer/layer_7/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_7_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_7/self_attention/key/add/ReadVariableOp¦ *transformer/layer_7/self_attention/key/addAddV2=transformer/layer_7/self_attention/key/einsum/Einsum:output:0Atransformer/layer_7/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_7/self_attention/key/add¢ Etransformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_7/self_attention/value/einsum/EinsumEinsum9transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_7/self_attention/value/einsum/Einsum’ ;transformer/layer_7/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_7_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_7/self_attention/value/add/ReadVariableOp® ,transformer/layer_7/self_attention/value/addAddV2?transformer/layer_7/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_7/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_7/self_attention/value/addÄ 0transformer/layer_7/self_attention/einsum/EinsumEinsum.transformer/layer_7/self_attention/key/add:z:00transformer/layer_7/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_7/self_attention/einsum/Einsum™ (transformer/layer_7/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_7/self_attention/Mul/y‘ &transformer/layer_7/self_attention/MulMul9transformer/layer_7/self_attention/einsum/Einsum:output:01transformer/layer_7/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_7/self_attention/MulŅ Btransformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dimÄ >transformer/layer_7/self_attention/masked_softmax_7/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_7/self_attention/masked_softmax_7/ExpandDims» 9transformer/layer_7/self_attention/masked_softmax_7/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_7/self_attention/masked_softmax_7/sub/xŅ 7transformer/layer_7/self_attention/masked_softmax_7/subSubBtransformer/layer_7/self_attention/masked_softmax_7/sub/x:output:0Gtransformer/layer_7/self_attention/masked_softmax_7/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_7/self_attention/masked_softmax_7/sub» 9transformer/layer_7/self_attention/masked_softmax_7/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_7/self_attention/masked_softmax_7/mul/yĘ 7transformer/layer_7/self_attention/masked_softmax_7/mulMul;transformer/layer_7/self_attention/masked_softmax_7/sub:z:0Btransformer/layer_7/self_attention/masked_softmax_7/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_7/self_attention/masked_softmax_7/mul° 7transformer/layer_7/self_attention/masked_softmax_7/addAddV2*transformer/layer_7/self_attention/Mul:z:0;transformer/layer_7/self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_7/self_attention/masked_softmax_7/addŽ ;transformer/layer_7/self_attention/masked_softmax_7/SoftmaxSoftmax;transformer/layer_7/self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_7/self_attention/masked_softmax_7/Softmax 6transformer/layer_7/self_attention/dropout_21/IdentityIdentityEtransformer/layer_7/self_attention/masked_softmax_7/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’28 6transformer/layer_7/self_attention/dropout_21/IdentityŠ 2transformer/layer_7/self_attention/einsum_1/EinsumEinsum?transformer/layer_7/self_attention/dropout_21/Identity:output:00transformer/layer_7/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_7/self_attention/einsum_1/EinsumĆ Ptransformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_7/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_7/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_7/self_attention/attention_output/einsum/Einsum Ftransformer/layer_7/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_7_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_7/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_7/self_attention/attention_output/addAddV2Jtransformer/layer_7/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_7/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_7/self_attention/attention_output/addŪ 'transformer/layer_7/dropout_22/IdentityIdentity;transformer/layer_7/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_7/dropout_22/Identityč transformer/layer_7/addAddV29transformer/layer_6/output_layer_norm/batchnorm/add_1:z:00transformer/layer_7/dropout_22/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_7/addę Ltransformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_7/self_attention_layer_norm/moments/meanMeantransformer/layer_7/add:z:0Utransformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_7/self_attention_layer_norm/moments/meanœ Btransformer/layer_7/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_7/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_7/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_7/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_7/add:z:0Ktransformer/layer_7/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_7/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_7/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_7/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_7/self_attention_layer_norm/moments/varianceĆ =transformer/layer_7/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_7/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_7/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_7/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_7/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_7/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_7/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_7/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_7/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_7/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_7/add:z:0?transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_7/self_attention_layer_norm/moments/mean:output:0?transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_7/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_7/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_7/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_7/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_7/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_7/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_7/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_7_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_7/intermediate/einsum/EinsumEinsumAtransformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_7/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_7/intermediate/einsum/Einsumä 3transformer/layer_7/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_7/output/einsum/EinsumŅ -transformer/layer_7/output/add/ReadVariableOpReadVariableOp6transformer_layer_7_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_7/output/add/ReadVariableOpó transformer/layer_7/output/addAddV21transformer/layer_7/output/einsum/Einsum:output:05transformer/layer_7/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_7/output/addĀ 'transformer/layer_7/dropout_23/IdentityIdentity"transformer/layer_7/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_7/dropout_23/Identityō transformer/layer_7/add_1AddV20transformer/layer_7/dropout_23/Identity:output:0Atransformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_7/add_1Ö Dtransformer/layer_7/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_7/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_7/output_layer_norm/moments/meanMeantransformer/layer_7/add_1:z:0Mtransformer/layer_7/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_7/output_layer_norm/moments/mean„ :transformer/layer_7/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_7/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_7/output_layer_norm/moments/StopGradient» ?transformer/layer_7/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_7/add_1:z:0Ctransformer/layer_7/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_7/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_7/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_7/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_7/output_layer_norm/moments/varianceMeanCtransformer/layer_7/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_7/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_7/output_layer_norm/moments/variance³ 5transformer/layer_7/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_7/output_layer_norm/batchnorm/add/y³ 3transformer/layer_7/output_layer_norm/batchnorm/addAddV2?transformer/layer_7/output_layer_norm/moments/variance:output:0>transformer/layer_7/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_7/output_layer_norm/batchnorm/addļ 5transformer/layer_7/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_7/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_7/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_7/output_layer_norm/batchnorm/mulMul9transformer/layer_7/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_7/output_layer_norm/batchnorm/mul 5transformer/layer_7/output_layer_norm/batchnorm/mul_1Multransformer/layer_7/add_1:z:07transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_7/output_layer_norm/batchnorm/mul_1« 5transformer/layer_7/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_7/output_layer_norm/moments/mean:output:07transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_7/output_layer_norm/batchnorm/mul_2… >transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_7_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_7/output_layer_norm/batchnorm/subSubFtransformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_7/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_7/output_layer_norm/batchnorm/sub« 5transformer/layer_7/output_layer_norm/batchnorm/add_1AddV29transformer/layer_7/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_7/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_7/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_8/self_attention/query/einsum/EinsumEinsum9transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_8/self_attention/query/einsum/Einsum’ ;transformer/layer_8/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_8_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_8/self_attention/query/add/ReadVariableOp® ,transformer/layer_8/self_attention/query/addAddV2?transformer/layer_8/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_8/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_8/self_attention/query/addœ Ctransformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_8/self_attention/key/einsum/EinsumEinsum9transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_8/self_attention/key/einsum/Einsumł 9transformer/layer_8/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_8_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_8/self_attention/key/add/ReadVariableOp¦ *transformer/layer_8/self_attention/key/addAddV2=transformer/layer_8/self_attention/key/einsum/Einsum:output:0Atransformer/layer_8/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_8/self_attention/key/add¢ Etransformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_8/self_attention/value/einsum/EinsumEinsum9transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_8/self_attention/value/einsum/Einsum’ ;transformer/layer_8/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_8_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_8/self_attention/value/add/ReadVariableOp® ,transformer/layer_8/self_attention/value/addAddV2?transformer/layer_8/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_8/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_8/self_attention/value/addÄ 0transformer/layer_8/self_attention/einsum/EinsumEinsum.transformer/layer_8/self_attention/key/add:z:00transformer/layer_8/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_8/self_attention/einsum/Einsum™ (transformer/layer_8/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_8/self_attention/Mul/y‘ &transformer/layer_8/self_attention/MulMul9transformer/layer_8/self_attention/einsum/Einsum:output:01transformer/layer_8/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_8/self_attention/MulŅ Btransformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dimÄ >transformer/layer_8/self_attention/masked_softmax_8/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_8/self_attention/masked_softmax_8/ExpandDims» 9transformer/layer_8/self_attention/masked_softmax_8/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_8/self_attention/masked_softmax_8/sub/xŅ 7transformer/layer_8/self_attention/masked_softmax_8/subSubBtransformer/layer_8/self_attention/masked_softmax_8/sub/x:output:0Gtransformer/layer_8/self_attention/masked_softmax_8/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_8/self_attention/masked_softmax_8/sub» 9transformer/layer_8/self_attention/masked_softmax_8/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_8/self_attention/masked_softmax_8/mul/yĘ 7transformer/layer_8/self_attention/masked_softmax_8/mulMul;transformer/layer_8/self_attention/masked_softmax_8/sub:z:0Btransformer/layer_8/self_attention/masked_softmax_8/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_8/self_attention/masked_softmax_8/mul° 7transformer/layer_8/self_attention/masked_softmax_8/addAddV2*transformer/layer_8/self_attention/Mul:z:0;transformer/layer_8/self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_8/self_attention/masked_softmax_8/addŽ ;transformer/layer_8/self_attention/masked_softmax_8/SoftmaxSoftmax;transformer/layer_8/self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_8/self_attention/masked_softmax_8/Softmax 6transformer/layer_8/self_attention/dropout_24/IdentityIdentityEtransformer/layer_8/self_attention/masked_softmax_8/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’28 6transformer/layer_8/self_attention/dropout_24/IdentityŠ 2transformer/layer_8/self_attention/einsum_1/EinsumEinsum?transformer/layer_8/self_attention/dropout_24/Identity:output:00transformer/layer_8/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_8/self_attention/einsum_1/EinsumĆ Ptransformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_8/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_8/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_8/self_attention/attention_output/einsum/Einsum Ftransformer/layer_8/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_8_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_8/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_8/self_attention/attention_output/addAddV2Jtransformer/layer_8/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_8/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_8/self_attention/attention_output/addŪ 'transformer/layer_8/dropout_25/IdentityIdentity;transformer/layer_8/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_8/dropout_25/Identityč transformer/layer_8/addAddV29transformer/layer_7/output_layer_norm/batchnorm/add_1:z:00transformer/layer_8/dropout_25/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_8/addę Ltransformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_8/self_attention_layer_norm/moments/meanMeantransformer/layer_8/add:z:0Utransformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_8/self_attention_layer_norm/moments/meanœ Btransformer/layer_8/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_8/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_8/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_8/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_8/add:z:0Ktransformer/layer_8/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_8/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_8/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_8/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_8/self_attention_layer_norm/moments/varianceĆ =transformer/layer_8/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_8/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_8/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_8/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_8/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_8/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_8/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_8/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_8/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_8/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_8/add:z:0?transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_8/self_attention_layer_norm/moments/mean:output:0?transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_8/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_8/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_8/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_8/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_8/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_8/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_8/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_8_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_8/intermediate/einsum/EinsumEinsumAtransformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_8/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_8/intermediate/einsum/Einsumä 3transformer/layer_8/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_8/output/einsum/EinsumŅ -transformer/layer_8/output/add/ReadVariableOpReadVariableOp6transformer_layer_8_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_8/output/add/ReadVariableOpó transformer/layer_8/output/addAddV21transformer/layer_8/output/einsum/Einsum:output:05transformer/layer_8/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_8/output/addĀ 'transformer/layer_8/dropout_26/IdentityIdentity"transformer/layer_8/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_8/dropout_26/Identityō transformer/layer_8/add_1AddV20transformer/layer_8/dropout_26/Identity:output:0Atransformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_8/add_1Ö Dtransformer/layer_8/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_8/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_8/output_layer_norm/moments/meanMeantransformer/layer_8/add_1:z:0Mtransformer/layer_8/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_8/output_layer_norm/moments/mean„ :transformer/layer_8/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_8/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_8/output_layer_norm/moments/StopGradient» ?transformer/layer_8/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_8/add_1:z:0Ctransformer/layer_8/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_8/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_8/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_8/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_8/output_layer_norm/moments/varianceMeanCtransformer/layer_8/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_8/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_8/output_layer_norm/moments/variance³ 5transformer/layer_8/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_8/output_layer_norm/batchnorm/add/y³ 3transformer/layer_8/output_layer_norm/batchnorm/addAddV2?transformer/layer_8/output_layer_norm/moments/variance:output:0>transformer/layer_8/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_8/output_layer_norm/batchnorm/addļ 5transformer/layer_8/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_8/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_8/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_8/output_layer_norm/batchnorm/mulMul9transformer/layer_8/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_8/output_layer_norm/batchnorm/mul 5transformer/layer_8/output_layer_norm/batchnorm/mul_1Multransformer/layer_8/add_1:z:07transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_8/output_layer_norm/batchnorm/mul_1« 5transformer/layer_8/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_8/output_layer_norm/moments/mean:output:07transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_8/output_layer_norm/batchnorm/mul_2… >transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_8_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_8/output_layer_norm/batchnorm/subSubFtransformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_8/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_8/output_layer_norm/batchnorm/sub« 5transformer/layer_8/output_layer_norm/batchnorm/add_1AddV29transformer/layer_8/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_8/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_8/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_9/self_attention/query/einsum/EinsumEinsum9transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_9/self_attention/query/einsum/Einsum’ ;transformer/layer_9/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_9_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_9/self_attention/query/add/ReadVariableOp® ,transformer/layer_9/self_attention/query/addAddV2?transformer/layer_9/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_9/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_9/self_attention/query/addœ Ctransformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_9/self_attention/key/einsum/EinsumEinsum9transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_9/self_attention/key/einsum/Einsumł 9transformer/layer_9/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_9_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_9/self_attention/key/add/ReadVariableOp¦ *transformer/layer_9/self_attention/key/addAddV2=transformer/layer_9/self_attention/key/einsum/Einsum:output:0Atransformer/layer_9/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_9/self_attention/key/add¢ Etransformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_9/self_attention/value/einsum/EinsumEinsum9transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_9/self_attention/value/einsum/Einsum’ ;transformer/layer_9/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_9_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_9/self_attention/value/add/ReadVariableOp® ,transformer/layer_9/self_attention/value/addAddV2?transformer/layer_9/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_9/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_9/self_attention/value/addÄ 0transformer/layer_9/self_attention/einsum/EinsumEinsum.transformer/layer_9/self_attention/key/add:z:00transformer/layer_9/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_9/self_attention/einsum/Einsum™ (transformer/layer_9/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_9/self_attention/Mul/y‘ &transformer/layer_9/self_attention/MulMul9transformer/layer_9/self_attention/einsum/Einsum:output:01transformer/layer_9/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_9/self_attention/MulŅ Btransformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dimÄ >transformer/layer_9/self_attention/masked_softmax_9/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_9/self_attention/masked_softmax_9/ExpandDims» 9transformer/layer_9/self_attention/masked_softmax_9/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_9/self_attention/masked_softmax_9/sub/xŅ 7transformer/layer_9/self_attention/masked_softmax_9/subSubBtransformer/layer_9/self_attention/masked_softmax_9/sub/x:output:0Gtransformer/layer_9/self_attention/masked_softmax_9/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_9/self_attention/masked_softmax_9/sub» 9transformer/layer_9/self_attention/masked_softmax_9/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_9/self_attention/masked_softmax_9/mul/yĘ 7transformer/layer_9/self_attention/masked_softmax_9/mulMul;transformer/layer_9/self_attention/masked_softmax_9/sub:z:0Btransformer/layer_9/self_attention/masked_softmax_9/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_9/self_attention/masked_softmax_9/mul° 7transformer/layer_9/self_attention/masked_softmax_9/addAddV2*transformer/layer_9/self_attention/Mul:z:0;transformer/layer_9/self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_9/self_attention/masked_softmax_9/addŽ ;transformer/layer_9/self_attention/masked_softmax_9/SoftmaxSoftmax;transformer/layer_9/self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_9/self_attention/masked_softmax_9/Softmax 6transformer/layer_9/self_attention/dropout_27/IdentityIdentityEtransformer/layer_9/self_attention/masked_softmax_9/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’28 6transformer/layer_9/self_attention/dropout_27/IdentityŠ 2transformer/layer_9/self_attention/einsum_1/EinsumEinsum?transformer/layer_9/self_attention/dropout_27/Identity:output:00transformer/layer_9/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_9/self_attention/einsum_1/EinsumĆ Ptransformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_9/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_9/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_9/self_attention/attention_output/einsum/Einsum Ftransformer/layer_9/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_9_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_9/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_9/self_attention/attention_output/addAddV2Jtransformer/layer_9/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_9/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_9/self_attention/attention_output/addŪ 'transformer/layer_9/dropout_28/IdentityIdentity;transformer/layer_9/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_9/dropout_28/Identityč transformer/layer_9/addAddV29transformer/layer_8/output_layer_norm/batchnorm/add_1:z:00transformer/layer_9/dropout_28/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_9/addę Ltransformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_9/self_attention_layer_norm/moments/meanMeantransformer/layer_9/add:z:0Utransformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_9/self_attention_layer_norm/moments/meanœ Btransformer/layer_9/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_9/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_9/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_9/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_9/add:z:0Ktransformer/layer_9/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_9/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_9/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_9/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_9/self_attention_layer_norm/moments/varianceĆ =transformer/layer_9/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_9/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_9/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_9/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_9/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_9/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_9/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_9/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_9/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_9/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_9/add:z:0?transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_9/self_attention_layer_norm/moments/mean:output:0?transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_9/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_9/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_9/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_9/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_9/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_9/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_9/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_9_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_9/intermediate/einsum/EinsumEinsumAtransformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_9/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_9/intermediate/einsum/Einsumä 3transformer/layer_9/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_9/output/einsum/EinsumŅ -transformer/layer_9/output/add/ReadVariableOpReadVariableOp6transformer_layer_9_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_9/output/add/ReadVariableOpó transformer/layer_9/output/addAddV21transformer/layer_9/output/einsum/Einsum:output:05transformer/layer_9/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_9/output/addĀ 'transformer/layer_9/dropout_29/IdentityIdentity"transformer/layer_9/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer/layer_9/dropout_29/Identityō transformer/layer_9/add_1AddV20transformer/layer_9/dropout_29/Identity:output:0Atransformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_9/add_1Ö Dtransformer/layer_9/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_9/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_9/output_layer_norm/moments/meanMeantransformer/layer_9/add_1:z:0Mtransformer/layer_9/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_9/output_layer_norm/moments/mean„ :transformer/layer_9/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_9/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_9/output_layer_norm/moments/StopGradient» ?transformer/layer_9/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_9/add_1:z:0Ctransformer/layer_9/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_9/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_9/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_9/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_9/output_layer_norm/moments/varianceMeanCtransformer/layer_9/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_9/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_9/output_layer_norm/moments/variance³ 5transformer/layer_9/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_9/output_layer_norm/batchnorm/add/y³ 3transformer/layer_9/output_layer_norm/batchnorm/addAddV2?transformer/layer_9/output_layer_norm/moments/variance:output:0>transformer/layer_9/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_9/output_layer_norm/batchnorm/addļ 5transformer/layer_9/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_9/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_9/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_9/output_layer_norm/batchnorm/mulMul9transformer/layer_9/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_9/output_layer_norm/batchnorm/mul 5transformer/layer_9/output_layer_norm/batchnorm/mul_1Multransformer/layer_9/add_1:z:07transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_9/output_layer_norm/batchnorm/mul_1« 5transformer/layer_9/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_9/output_layer_norm/moments/mean:output:07transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_9/output_layer_norm/batchnorm/mul_2… >transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_9_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_9/output_layer_norm/batchnorm/subSubFtransformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_9/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_9/output_layer_norm/batchnorm/sub« 5transformer/layer_9/output_layer_norm/batchnorm/add_1AddV29transformer/layer_9/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_9/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_9/output_layer_norm/batchnorm/add_1„ Ftransformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpOtransformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02H Ftransformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpš 7transformer/layer_10/self_attention/query/einsum/EinsumEinsum9transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0Ntransformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde29 7transformer/layer_10/self_attention/query/einsum/Einsum‚ abde27 5transformer/layer_10/self_attention/key/einsum/Einsumü :transformer/layer_10/self_attention/key/add/ReadVariableOpReadVariableOpCtransformer_layer_10_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02< :transformer/layer_10/self_attention/key/add/ReadVariableOpŖ +transformer/layer_10/self_attention/key/addAddV2>transformer/layer_10/self_attention/key/einsum/Einsum:output:0Btransformer/layer_10/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2- +transformer/layer_10/self_attention/key/add„ Ftransformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpOtransformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02H Ftransformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpš 7transformer/layer_10/self_attention/value/einsum/EinsumEinsum9transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0Ntransformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde29 7transformer/layer_10/self_attention/value/einsum/Einsum‚ acbe23 1transformer/layer_10/self_attention/einsum/Einsum› )transformer/layer_10/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2+ )transformer/layer_10/self_attention/Mul/y• 'transformer/layer_10/self_attention/MulMul:transformer/layer_10/self_attention/einsum/Einsum:output:02transformer/layer_10/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'transformer/layer_10/self_attention/MulÖ Dtransformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimŹ @transformer/layer_10/self_attention/masked_softmax_10/ExpandDims ExpandDimsself_attention_mask/mul:z:0Mtransformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2B @transformer/layer_10/self_attention/masked_softmax_10/ExpandDimsæ ;transformer/layer_10/self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2= ;transformer/layer_10/self_attention/masked_softmax_10/sub/xŚ 9transformer/layer_10/self_attention/masked_softmax_10/subSubDtransformer/layer_10/self_attention/masked_softmax_10/sub/x:output:0Itransformer/layer_10/self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_10/self_attention/masked_softmax_10/subæ ;transformer/layer_10/self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2= ;transformer/layer_10/self_attention/masked_softmax_10/mul/yĪ 9transformer/layer_10/self_attention/masked_softmax_10/mulMul=transformer/layer_10/self_attention/masked_softmax_10/sub:z:0Dtransformer/layer_10/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_10/self_attention/masked_softmax_10/mul· 9transformer/layer_10/self_attention/masked_softmax_10/addAddV2+transformer/layer_10/self_attention/Mul:z:0=transformer/layer_10/self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_10/self_attention/masked_softmax_10/add” =transformer/layer_10/self_attention/masked_softmax_10/SoftmaxSoftmax=transformer/layer_10/self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_10/self_attention/masked_softmax_10/Softmax“ 7transformer/layer_10/self_attention/dropout_30/IdentityIdentityGtransformer/layer_10/self_attention/masked_softmax_10/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_10/self_attention/dropout_30/IdentityŌ 3transformer/layer_10/self_attention/einsum_1/EinsumEinsum@transformer/layer_10/self_attention/dropout_30/Identity:output:01transformer/layer_10/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd25 3transformer/layer_10/self_attention/einsum_1/EinsumĘ Qtransformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02S Qtransformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ Btransformer/layer_10/self_attention/attention_output/einsum/EinsumEinsumabe2D Btransformer/layer_10/self_attention/attention_output/einsum/Einsum  Gtransformer/layer_10/self_attention/attention_output/add/ReadVariableOpReadVariableOpPtransformer_layer_10_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_10/self_attention/attention_output/add/ReadVariableOpŪ 8transformer/layer_10/self_attention/attention_output/addAddV2Ktransformer/layer_10/self_attention/attention_output/einsum/Einsum:output:0Otransformer/layer_10/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer/layer_10/self_attention/attention_output/addŽ (transformer/layer_10/dropout_31/IdentityIdentitytransformer/layer_10/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/add/y× transformer/layer_10/self_attention_layer_norm/batchnorm/RsqrtRsqrt@transformer/layer_10/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/Rsqrt¬ Ktransformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpTtransformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02M Ktransformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpÜ transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_10/add:z:0@transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Ļ >transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2MulDtransformer/layer_10/self_attention_layer_norm/moments/mean:output:0@transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2  Gtransformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpPtransformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpŲ transformer/layer_10/self_attention_layer_norm/batchnorm/add_1AddV2Btransformer/layer_10/self_attention_layer_norm/batchnorm/mul_1:z:0@transformer/layer_10/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/add_1Š >transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpGtransformer_layer_10_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02@ >transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpÜ /transformer/layer_10/intermediate/einsum/EinsumEinsumBtransformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0Ftransformer/layer_10/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd21 /transformer/layer_10/intermediate/einsum/Einsumē 4transformer/layer_10/intermediate/add/ReadVariableOpReadVariableOp=transformer_layer_10_intermediate_add_readvariableop_resource* _output_shapes :€* dtype026 4transformer/layer_10/intermediate/add/ReadVariableOp %transformer/layer_10/intermediate/addAddV28transformer/layer_10/intermediate/einsum/Einsum:output:0abd2+ )transformer/layer_10/output/einsum/EinsumÕ .transformer/layer_10/output/add/ReadVariableOpReadVariableOp7transformer_layer_10_output_add_readvariableop_resource* _output_shapes :€* dtype020 .transformer/layer_10/output/add/ReadVariableOp÷ transformer/layer_10/output/addAddV22transformer/layer_10/output/einsum/Einsum:output:06transformer/layer_10/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! transformer/layer_10/output/addÅ (transformer/layer_10/dropout_32/IdentityIdentity#transformer/layer_10/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2* (transformer/layer_10/dropout_32/Identityų transformer/layer_10/add_1AddV21transformer/layer_10/dropout_32/Identity:output:0Btransformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_10/add_1Ų Etransformer/layer_10/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2G Etransformer/layer_10/output_layer_norm/moments/mean/reduction_indices² 3transformer/layer_10/output_layer_norm/moments/meanMeantransformer/layer_10/add_1:z:0Ntransformer/layer_10/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(25 3transformer/layer_10/output_layer_norm/moments/mean‡ ;transformer/layer_10/output_layer_norm/moments/StopGradient StopGradientabde29 7transformer/layer_11/self_attention/query/einsum/Einsum‚ abde27 5transformer/layer_11/self_attention/key/einsum/Einsumü :transformer/layer_11/self_attention/key/add/ReadVariableOpReadVariableOpCtransformer_layer_11_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02< :transformer/layer_11/self_attention/key/add/ReadVariableOpŖ +transformer/layer_11/self_attention/key/addAddV2>transformer/layer_11/self_attention/key/einsum/Einsum:output:0Btransformer/layer_11/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2- +transformer/layer_11/self_attention/key/add„ Ftransformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpOtransformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02H Ftransformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpń 7transformer/layer_11/self_attention/value/einsum/EinsumEinsum:transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0Ntransformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde29 7transformer/layer_11/self_attention/value/einsum/Einsum‚ acbe23 1transformer/layer_11/self_attention/einsum/Einsum› )transformer/layer_11/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2+ )transformer/layer_11/self_attention/Mul/y• 'transformer/layer_11/self_attention/MulMul:transformer/layer_11/self_attention/einsum/Einsum:output:02transformer/layer_11/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'transformer/layer_11/self_attention/MulÖ Dtransformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimŹ @transformer/layer_11/self_attention/masked_softmax_11/ExpandDims ExpandDimsself_attention_mask/mul:z:0Mtransformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2B @transformer/layer_11/self_attention/masked_softmax_11/ExpandDimsæ ;transformer/layer_11/self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2= ;transformer/layer_11/self_attention/masked_softmax_11/sub/xŚ 9transformer/layer_11/self_attention/masked_softmax_11/subSubDtransformer/layer_11/self_attention/masked_softmax_11/sub/x:output:0Itransformer/layer_11/self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_11/self_attention/masked_softmax_11/subæ ;transformer/layer_11/self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2= ;transformer/layer_11/self_attention/masked_softmax_11/mul/yĪ 9transformer/layer_11/self_attention/masked_softmax_11/mulMul=transformer/layer_11/self_attention/masked_softmax_11/sub:z:0Dtransformer/layer_11/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_11/self_attention/masked_softmax_11/mul· 9transformer/layer_11/self_attention/masked_softmax_11/addAddV2+transformer/layer_11/self_attention/Mul:z:0=transformer/layer_11/self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_11/self_attention/masked_softmax_11/add” =transformer/layer_11/self_attention/masked_softmax_11/SoftmaxSoftmax=transformer/layer_11/self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_11/self_attention/masked_softmax_11/Softmax“ 7transformer/layer_11/self_attention/dropout_33/IdentityIdentityGtransformer/layer_11/self_attention/masked_softmax_11/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_11/self_attention/dropout_33/IdentityŌ 3transformer/layer_11/self_attention/einsum_1/EinsumEinsum@transformer/layer_11/self_attention/dropout_33/Identity:output:01transformer/layer_11/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd25 3transformer/layer_11/self_attention/einsum_1/EinsumĘ Qtransformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02S Qtransformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ Btransformer/layer_11/self_attention/attention_output/einsum/EinsumEinsumabe2D Btransformer/layer_11/self_attention/attention_output/einsum/Einsum  Gtransformer/layer_11/self_attention/attention_output/add/ReadVariableOpReadVariableOpPtransformer_layer_11_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_11/self_attention/attention_output/add/ReadVariableOpŪ 8transformer/layer_11/self_attention/attention_output/addAddV2Ktransformer/layer_11/self_attention/attention_output/einsum/Einsum:output:0Otransformer/layer_11/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer/layer_11/self_attention/attention_output/addŽ (transformer/layer_11/dropout_34/IdentityIdentitytransformer/layer_11/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/add/y× transformer/layer_11/self_attention_layer_norm/batchnorm/RsqrtRsqrt@transformer/layer_11/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/Rsqrt¬ Ktransformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpTtransformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02M Ktransformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpÜ transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_11/add:z:0@transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Ļ >transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2MulDtransformer/layer_11/self_attention_layer_norm/moments/mean:output:0@transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2  Gtransformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpPtransformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpŲ transformer/layer_11/self_attention_layer_norm/batchnorm/add_1AddV2Btransformer/layer_11/self_attention_layer_norm/batchnorm/mul_1:z:0@transformer/layer_11/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/add_1Š >transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpGtransformer_layer_11_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02@ >transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpÜ /transformer/layer_11/intermediate/einsum/EinsumEinsumBtransformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0Ftransformer/layer_11/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd21 /transformer/layer_11/intermediate/einsum/Einsumē 4transformer/layer_11/intermediate/add/ReadVariableOpReadVariableOp=transformer_layer_11_intermediate_add_readvariableop_resource* _output_shapes :€* dtype026 4transformer/layer_11/intermediate/add/ReadVariableOp %transformer/layer_11/intermediate/addAddV28transformer/layer_11/intermediate/einsum/Einsum:output:0abd2+ )transformer/layer_11/output/einsum/EinsumÕ .transformer/layer_11/output/add/ReadVariableOpReadVariableOp7transformer_layer_11_output_add_readvariableop_resource* _output_shapes :€* dtype020 .transformer/layer_11/output/add/ReadVariableOp÷ transformer/layer_11/output/addAddV22transformer/layer_11/output/einsum/Einsum:output:06transformer/layer_11/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! transformer/layer_11/output/addÅ (transformer/layer_11/dropout_35/IdentityIdentity#transformer/layer_11/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2* (transformer/layer_11/dropout_35/Identityų transformer/layer_11/add_1AddV21transformer/layer_11/dropout_35/Identity:output:0Btransformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_11/add_1Ų Etransformer/layer_11/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2G Etransformer/layer_11/output_layer_norm/moments/mean/reduction_indices² 3transformer/layer_11/output_layer_norm/moments/meanMeantransformer/layer_11/add_1:z:0Ntransformer/layer_11/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(25 3transformer/layer_11/output_layer_norm/moments/mean‡ ;transformer/layer_11/output_layer_norm/moments/StopGradient StopGradient  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ųŗ ” N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_10387 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_7/ExpandDims/dimõ *self_attention/masked_softmax_7/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_7/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_7/ExpandDims“ %self_attention/masked_softmax_7/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_7/sub/x‚ #self_attention/masked_softmax_7/subSub.self_attention/masked_softmax_7/sub/x:output:03self_attention/masked_softmax_7/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/sub“ %self_attention/masked_softmax_7/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_7/mul/yö #self_attention/masked_softmax_7/mulMul'self_attention/masked_softmax_7/sub:z:0.self_attention/masked_softmax_7/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/mulą #self_attention/masked_softmax_7/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/addŅ 'self_attention/masked_softmax_7/SoftmaxSoftmax'self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_7/Softmax— 'self_attention/dropout_21/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_21/dropout/Const† %self_attention/dropout_21/dropout/MulMul1self_attention/masked_softmax_7/Softmax:softmax:00self_attention/dropout_21/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_21/dropout/Mul³ 'self_attention/dropout_21/dropout/ShapeShape1self_attention/masked_softmax_7/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_21/dropout/Shapeœ >self_attention/dropout_21/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_21/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_21/dropout/random_uniform/RandomUniform© 0self_attention/dropout_21/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_21/dropout/GreaterEqual/yĄ .self_attention/dropout_21/dropout/GreaterEqual GreaterEqualGself_attention/dropout_21/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_21/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_21/dropout/GreaterEqualē &self_attention/dropout_21/dropout/CastCast2self_attention/dropout_21/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_21/dropout/Castü 'self_attention/dropout_21/dropout/Mul_1Mul)self_attention/dropout_21/dropout/Mul:z:0*self_attention/dropout_21/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_21/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_21/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_22/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_22/dropout/ConstĆ dropout_22/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_22/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/dropout/Mul‹ dropout_22/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_22/dropout/Shapeć /dropout_22/dropout/random_uniform/RandomUniform RandomUniform!dropout_22/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_22/dropout/random_uniform/RandomUniform‹ !dropout_22/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_22/dropout/GreaterEqual/yų dropout_22/dropout/GreaterEqual GreaterEqual8dropout_22/dropout/random_uniform/RandomUniform:output:0*dropout_22/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_22/dropout/GreaterEqual® dropout_22/dropout/CastCast#dropout_22/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/dropout/Cast“ dropout_22/dropout/Mul_1Muldropout_22/dropout/Mul:z:0dropout_22/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/dropout/Mul_1y addAddV2inputsdropout_22/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_7/Pow/yž activation_7/PowPowintermediate/add:z:0activation_7/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Powm activation_7/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_7/mul/xž activation_7/mulMulactivation_7/mul/x:output:0activation_7/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul™ activation_7/addAddV2intermediate/add:z:0activation_7/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/addq activation_7/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_7/mul_1/x¤ activation_7/mul_1Mulactivation_7/mul_1/x:output:0activation_7/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_1† activation_7/TanhTanhactivation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Tanhq activation_7/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_7/add_1/x§ activation_7/add_1AddV2activation_7/add_1/x:output:0activation_7/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/add_1q activation_7/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_7/mul_2/x¦ activation_7/mul_2Mulactivation_7/mul_2/x:output:0activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_2 activation_7/mul_3Mulintermediate/add:z:0activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_7/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_23/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_23/dropout/ConstŖ dropout_23/dropout/MulMuloutput/add:z:0!dropout_23/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/dropout/Mulr dropout_23/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_23/dropout/Shapeć /dropout_23/dropout/random_uniform/RandomUniform RandomUniform!dropout_23/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_23/dropout/random_uniform/RandomUniform‹ !dropout_23/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_23/dropout/GreaterEqual/yų dropout_23/dropout/GreaterEqual GreaterEqual8dropout_23/dropout/random_uniform/RandomUniform:output:0*dropout_23/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_23/dropout/GreaterEqual® dropout_23/dropout/CastCast#dropout_23/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/dropout/Cast“ dropout_23/dropout/Mul_1Muldropout_23/dropout/Mul:z:0dropout_23/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/dropout/Mul_1¤ add_1AddV2dropout_23/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ļp ß 3__inference_transformer_encoder_layer_call_fn_24904 inputs_0 inputs_1 inputs_2 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall§ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1inputs_2unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_129062 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity“ Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/2:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : įp Ų ,__inference_functional_1_layer_call_fn_21153 inputs_0 inputs_1 inputs_2 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall  StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1inputs_2unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*P fKRI G__inference_functional_1_layer_call_and_return_conditional_losses_161832 StatefulPartitionedCall IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity  Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/2:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : Ē» ¤ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_29052 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_11/ExpandDims/dimų +self_attention/masked_softmax_11/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_11/ExpandDims• &self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_11/sub/x† $self_attention/masked_softmax_11/subSub/self_attention/masked_softmax_11/sub/x:output:04self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/sub• &self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_11/mul/yś $self_attention/masked_softmax_11/mulMul(self_attention/masked_softmax_11/sub:z:0/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/mulć $self_attention/masked_softmax_11/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/addÕ (self_attention/masked_softmax_11/SoftmaxSoftmax(self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_11/Softmax— 'self_attention/dropout_33/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_33/dropout/Const‡ %self_attention/dropout_33/dropout/MulMul2self_attention/masked_softmax_11/Softmax:softmax:00self_attention/dropout_33/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_33/dropout/Mul“ 'self_attention/dropout_33/dropout/ShapeShape2self_attention/masked_softmax_11/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_33/dropout/Shapeœ >self_attention/dropout_33/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_33/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_33/dropout/random_uniform/RandomUniform© 0self_attention/dropout_33/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_33/dropout/GreaterEqual/yĄ .self_attention/dropout_33/dropout/GreaterEqual GreaterEqualGself_attention/dropout_33/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_33/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_33/dropout/GreaterEqualē &self_attention/dropout_33/dropout/CastCast2self_attention/dropout_33/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_33/dropout/Castü 'self_attention/dropout_33/dropout/Mul_1Mul)self_attention/dropout_33/dropout/Mul:z:0*self_attention/dropout_33/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_33/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_33/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_34/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_34/dropout/ConstĆ dropout_34/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_34/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/dropout/Mul‹ dropout_34/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_34/dropout/Shapeć /dropout_34/dropout/random_uniform/RandomUniform RandomUniform!dropout_34/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_34/dropout/random_uniform/RandomUniform‹ !dropout_34/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_34/dropout/GreaterEqual/yų dropout_34/dropout/GreaterEqual GreaterEqual8dropout_34/dropout/random_uniform/RandomUniform:output:0*dropout_34/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_34/dropout/GreaterEqual® dropout_34/dropout/CastCast#dropout_34/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/dropout/Cast“ dropout_34/dropout/Mul_1Muldropout_34/dropout/Mul:z:0dropout_34/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/dropout/Mul_1{ addAddV2inputs_0dropout_34/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_11/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_11/Pow/y” activation_11/PowPowintermediate/add:z:0activation_11/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Powo activation_11/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_11/mul/x¢ activation_11/mulMulactivation_11/mul/x:output:0activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mulœ activation_11/addAddV2intermediate/add:z:0activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/adds activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_11/mul_1/xØ activation_11/mul_1Mulactivation_11/mul_1/x:output:0activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_1‰ activation_11/TanhTanhactivation_11/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Tanhs activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_11/add_1/x« activation_11/add_1AddV2activation_11/add_1/x:output:0activation_11/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/add_1s activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_11/mul_2/xŖ activation_11/mul_2Mulactivation_11/mul_2/x:output:0activation_11/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_2  activation_11/mul_3Mulintermediate/add:z:0activation_11/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_11/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_35/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_35/dropout/ConstŖ dropout_35/dropout/MulMuloutput/add:z:0!dropout_35/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/dropout/Mulr dropout_35/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_35/dropout/Shapeć /dropout_35/dropout/random_uniform/RandomUniform RandomUniform!dropout_35/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_35/dropout/random_uniform/RandomUniform‹ !dropout_35/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_35/dropout/GreaterEqual/yų dropout_35/dropout/GreaterEqual GreaterEqual8dropout_35/dropout/random_uniform/RandomUniform:output:0*dropout_35/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_35/dropout/GreaterEqual® dropout_35/dropout/CastCast#dropout_35/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/dropout/Cast“ dropout_35/dropout/Mul_1Muldropout_35/dropout/Mul:z:0dropout_35/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/dropout/Mul_1¤ add_1AddV2dropout_35/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_0_layer_call_fn_25814 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_80722 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : „ ļ 3__inference_transformer/layer_9_layer_call_fn_28575 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_111862 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : •š £ N__inference_transformer/layer_1_layer_call_and_return_conditional_losses_26049 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_1/ExpandDims/dimõ *self_attention/masked_softmax_1/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_1/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_1/ExpandDims“ %self_attention/masked_softmax_1/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_1/sub/x‚ #self_attention/masked_softmax_1/subSub.self_attention/masked_softmax_1/sub/x:output:03self_attention/masked_softmax_1/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/sub“ %self_attention/masked_softmax_1/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_1/mul/yö #self_attention/masked_softmax_1/mulMul'self_attention/masked_softmax_1/sub:z:0.self_attention/masked_softmax_1/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/mulą #self_attention/masked_softmax_1/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/addŅ 'self_attention/masked_softmax_1/SoftmaxSoftmax'self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_1/SoftmaxŃ !self_attention/dropout_3/IdentityIdentity1self_attention/masked_softmax_1/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/dropout_3/Identity’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_3/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/add dropout_4/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/Identityz addAddV2inputs_0dropout_4/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_1/Pow/yž activation_1/PowPowintermediate/add:z:0activation_1/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Powm activation_1/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_1/mul/xž activation_1/mulMulactivation_1/mul/x:output:0activation_1/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul™ activation_1/addAddV2intermediate/add:z:0activation_1/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/addq activation_1/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_1/mul_1/x¤ activation_1/mul_1Mulactivation_1/mul_1/x:output:0activation_1/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_1† activation_1/TanhTanhactivation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Tanhq activation_1/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_1/add_1/x§ activation_1/add_1AddV2activation_1/add_1/x:output:0activation_1/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/add_1q activation_1/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_1/mul_2/x¦ activation_1/mul_2Mulactivation_1/mul_2/x:output:0activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_2 activation_1/mul_3Mulintermediate/add:z:0activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_1/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add„ dropout_5/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/Identity£ add_1AddV2dropout_5/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_4_layer_call_fn_27058 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_94562 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ż ` B__inference_dropout_layer_call_and_return_conditional_losses_25450 inputs identity_1h IdentityIdentityinputs* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identityw Identity_1IdentityIdentity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1"! identity_1Identity_1:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs å Š 5__inference_embeddings/layer_norm_layer_call_fn_25433 inputs unknown unknown_0 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputsunknown unknown_0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*X fSRQ O__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_77452 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*< _input_shapes+ ): ’’’’’’’’’ ’’’’’’’’’€::22 StatefulPartitionedCallStatefulPartitionedCall:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_0_layer_call_fn_25776 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_80722 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : įp Ų ,__inference_functional_1_layer_call_fn_21560 inputs_0 inputs_1 inputs_2 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall  StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1inputs_2unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*P fKRI G__inference_functional_1_layer_call_and_return_conditional_losses_169972 StatefulPartitionedCall IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity  Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/2:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ™q ķ 3__inference_transformer_encoder_layer_call_fn_14142 input_word_ids input_mask input_type_ids unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCallµ StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idsunknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_137392 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity“ Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ƒ ļ 3__inference_transformer/layer_1_layer_call_fn_26125 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_84182 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ķ ‰ M__inference_position_embedding_layer_call_and_return_conditional_losses_25355 inputs readvariableop_resource identityˆD ShapeShapeinputs* T0* _output_shapes :2 Shapet strided_slice/stackConst* _output_shapes :* dtype0* value B :2 strided_slice/stackx strided_slice/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_1x strided_slice/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_2ā strided_slice StridedSliceShape:output:0strided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slicex strided_slice_1/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_1/stack| strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_1| strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_2ģ strided_slice_1 StridedSliceShape:output:0strided_slice_1/stack:output:0 strided_slice_1/stack_1:output:0 strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_1z ReadVariableOpReadVariableOpreadvariableop_resource* _output_shapes : €€* dtype02 ReadVariableOp strided_slice_2/stackConst* _output_shapes :* dtype0* valueB"2 strided_slice_2/stackx strided_slice_2/stack_1/1Const* _output_shapes :* dtype0* value B:2 strided_slice_2/stack_1/1¦ strided_slice_2/stack_1Packstrided_slice_1:output:0"strided_slice_2/stack_1/1:output:0* N* T0* _output_shapes :2 strided_slice_2/stack_1ƒ strided_slice_2/stack_2Const* _output_shapes :* dtype0* valueB"2 strided_slice_2/stack_2 strided_slice_2 StridedSliceReadVariableOp:value:0strided_slice_2/stack:output:0 strided_slice_2/stack_1:output:0 strided_slice_2/stack_2:output:0* Index0* T0*( _output_shapes : ’’’’’’’’’€* begin_mask* end_mask2 strided_slice_2m BroadcastTo/shape/2Const* _output_shapes :* dtype0* value B:€2 BroadcastTo/shape/2¬ BroadcastTo/shapePackstrided_slice:output:0strided_slice_1:output:0BroadcastTo/shape/2:output:0* N* T0* _output_shapes :2 BroadcastTo/shapeŸ BroadcastTo BroadcastTostrided_slice_2:output:0BroadcastTo/shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 BroadcastTov IdentityIdentityBroadcastTo:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*8 _input_shapes' %: ’’’’’’’’’ ’’’’’’’’’€::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes : Æŗ   M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_8311 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_1/ExpandDims/dimõ *self_attention/masked_softmax_1/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_1/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_1/ExpandDims“ %self_attention/masked_softmax_1/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_1/sub/x‚ #self_attention/masked_softmax_1/subSub.self_attention/masked_softmax_1/sub/x:output:03self_attention/masked_softmax_1/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/sub“ %self_attention/masked_softmax_1/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_1/mul/yö #self_attention/masked_softmax_1/mulMul'self_attention/masked_softmax_1/sub:z:0.self_attention/masked_softmax_1/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/mulą #self_attention/masked_softmax_1/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/addŅ 'self_attention/masked_softmax_1/SoftmaxSoftmax'self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_1/Softmax• &self_attention/dropout_3/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2( &self_attention/dropout_3/dropout/Constƒ $self_attention/dropout_3/dropout/MulMul1self_attention/masked_softmax_1/Softmax:softmax:0/self_attention/dropout_3/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout_3/dropout/Mul± &self_attention/dropout_3/dropout/ShapeShape1self_attention/masked_softmax_1/Softmax:softmax:0* T0* _output_shapes :2( &self_attention/dropout_3/dropout/Shape™ =self_attention/dropout_3/dropout/random_uniform/RandomUniform RandomUniform/self_attention/dropout_3/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02? =self_attention/dropout_3/dropout/random_uniform/RandomUniform§ /self_attention/dropout_3/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=21 /self_attention/dropout_3/dropout/GreaterEqual/y¼ -self_attention/dropout_3/dropout/GreaterEqual GreaterEqualFself_attention/dropout_3/dropout/random_uniform/RandomUniform:output:08self_attention/dropout_3/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2/ -self_attention/dropout_3/dropout/GreaterEqualä %self_attention/dropout_3/dropout/CastCast1self_attention/dropout_3/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_3/dropout/Castų &self_attention/dropout_3/dropout/Mul_1Mul(self_attention/dropout_3/dropout/Mul:z:0)self_attention/dropout_3/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_3/dropout/Mul_1’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_3/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addw dropout_4/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_4/dropout/ConstĄ dropout_4/dropout/MulMul'self_attention/attention_output/add:z:0 dropout_4/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/Mul‰ dropout_4/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_4/dropout/Shapeą .dropout_4/dropout/random_uniform/RandomUniform RandomUniform dropout_4/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_4/dropout/random_uniform/RandomUniform‰ dropout_4/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_4/dropout/GreaterEqual/yō dropout_4/dropout/GreaterEqual GreaterEqual7dropout_4/dropout/random_uniform/RandomUniform:output:0)dropout_4/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/GreaterEqual« dropout_4/dropout/CastCast"dropout_4/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/Cast° dropout_4/dropout/Mul_1Muldropout_4/dropout/Mul:z:0dropout_4/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/Mul_1x addAddV2inputsdropout_4/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_1/Pow/yž activation_1/PowPowintermediate/add:z:0activation_1/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Powm activation_1/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_1/mul/xž activation_1/mulMulactivation_1/mul/x:output:0activation_1/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul™ activation_1/addAddV2intermediate/add:z:0activation_1/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/addq activation_1/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_1/mul_1/x¤ activation_1/mul_1Mulactivation_1/mul_1/x:output:0activation_1/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_1† activation_1/TanhTanhactivation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Tanhq activation_1/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_1/add_1/x§ activation_1/add_1AddV2activation_1/add_1/x:output:0activation_1/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/add_1q activation_1/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_1/mul_2/x¦ activation_1/mul_2Mulactivation_1/mul_2/x:output:0activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_2 activation_1/mul_3Mulintermediate/add:z:0activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_1/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addw dropout_5/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_5/dropout/Const§ dropout_5/dropout/MulMuloutput/add:z:0 dropout_5/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/Mulp dropout_5/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_5/dropout/Shapeą .dropout_5/dropout/random_uniform/RandomUniform RandomUniform dropout_5/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_5/dropout/random_uniform/RandomUniform‰ dropout_5/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_5/dropout/GreaterEqual/yō dropout_5/dropout/GreaterEqual GreaterEqual7dropout_5/dropout/random_uniform/RandomUniform:output:0)dropout_5/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/GreaterEqual« dropout_5/dropout/CastCast"dropout_5/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/Cast° dropout_5/dropout/Mul_1Muldropout_5/dropout/Mul:z:0dropout_5/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/Mul_1£ add_1AddV2dropout_5/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : К ¢ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_11532 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_10/ExpandDims/dimų +self_attention/masked_softmax_10/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_10/ExpandDims• &self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_10/sub/x† $self_attention/masked_softmax_10/subSub/self_attention/masked_softmax_10/sub/x:output:04self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/sub• &self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_10/mul/yś $self_attention/masked_softmax_10/mulMul(self_attention/masked_softmax_10/sub:z:0/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/mulć $self_attention/masked_softmax_10/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/addÕ (self_attention/masked_softmax_10/SoftmaxSoftmax(self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_10/SoftmaxŌ "self_attention/dropout_30/IdentityIdentity2self_attention/masked_softmax_10/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_30/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_30/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_31/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/Identityy addAddV2inputsdropout_31/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_10/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_10/Pow/y” activation_10/PowPowintermediate/add:z:0activation_10/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Powo activation_10/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_10/mul/x¢ activation_10/mulMulactivation_10/mul/x:output:0activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mulœ activation_10/addAddV2intermediate/add:z:0activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/adds activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_10/mul_1/xØ activation_10/mul_1Mulactivation_10/mul_1/x:output:0activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_1‰ activation_10/TanhTanhactivation_10/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Tanhs activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_10/add_1/x« activation_10/add_1AddV2activation_10/add_1/x:output:0activation_10/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/add_1s activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_10/mul_2/xŖ activation_10/mul_2Mulactivation_10/mul_2/x:output:0activation_10/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_2  activation_10/mul_3Mulintermediate/add:z:0activation_10/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_10/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_32/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/Identity¤ add_1AddV2dropout_32/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_5_layer_call_fn_27331 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_98022 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ‰  é1 G__inference_functional_1_layer_call_and_return_conditional_losses_15364 input_word_ids input_mask input_type_ids transformer_encoder_14962 transformer_encoder_14964 transformer_encoder_14966 transformer_encoder_14968 transformer_encoder_14970 transformer_encoder_14972 transformer_encoder_14974 transformer_encoder_14976 transformer_encoder_14978 transformer_encoder_14980 transformer_encoder_14982 transformer_encoder_14984 transformer_encoder_14986 transformer_encoder_14988 transformer_encoder_14990 transformer_encoder_14992 transformer_encoder_14994 transformer_encoder_14996 transformer_encoder_14998 transformer_encoder_15000 transformer_encoder_15002 transformer_encoder_15004 transformer_encoder_15006 transformer_encoder_15008 transformer_encoder_15010 transformer_encoder_15012 transformer_encoder_15014 transformer_encoder_15016 transformer_encoder_15018 transformer_encoder_15020 transformer_encoder_15022 transformer_encoder_15024 transformer_encoder_15026 transformer_encoder_15028 transformer_encoder_15030 transformer_encoder_15032 transformer_encoder_15034 transformer_encoder_15036 transformer_encoder_15038 transformer_encoder_15040 transformer_encoder_15042 transformer_encoder_15044 transformer_encoder_15046 transformer_encoder_15048 transformer_encoder_15050 transformer_encoder_15052 transformer_encoder_15054 transformer_encoder_15056 transformer_encoder_15058 transformer_encoder_15060 transformer_encoder_15062 transformer_encoder_15064 transformer_encoder_15066 transformer_encoder_15068 transformer_encoder_15070 transformer_encoder_15072 transformer_encoder_15074 transformer_encoder_15076 transformer_encoder_15078 transformer_encoder_15080 transformer_encoder_15082 transformer_encoder_15084 transformer_encoder_15086 transformer_encoder_15088 transformer_encoder_15090 transformer_encoder_15092 transformer_encoder_15094 transformer_encoder_15096 transformer_encoder_15098 transformer_encoder_15100 transformer_encoder_15102 transformer_encoder_15104 transformer_encoder_15106 transformer_encoder_15108 transformer_encoder_15110 transformer_encoder_15112 transformer_encoder_15114 transformer_encoder_15116 transformer_encoder_15118 transformer_encoder_15120 transformer_encoder_15122 transformer_encoder_15124 transformer_encoder_15126 transformer_encoder_15128 transformer_encoder_15130 transformer_encoder_15132 transformer_encoder_15134 transformer_encoder_15136 transformer_encoder_15138 transformer_encoder_15140 transformer_encoder_15142 transformer_encoder_15144 transformer_encoder_15146 transformer_encoder_15148 transformer_encoder_15150 transformer_encoder_15152 transformer_encoder_15154 transformer_encoder_15156 transformer_encoder_15158 transformer_encoder_15160 transformer_encoder_15162 transformer_encoder_15164 transformer_encoder_15166 transformer_encoder_15168 transformer_encoder_15170 transformer_encoder_15172 transformer_encoder_15174 transformer_encoder_15176 transformer_encoder_15178 transformer_encoder_15180 transformer_encoder_15182 transformer_encoder_15184 transformer_encoder_15186 transformer_encoder_15188 transformer_encoder_15190 transformer_encoder_15192 transformer_encoder_15194 transformer_encoder_15196 transformer_encoder_15198 transformer_encoder_15200 transformer_encoder_15202 transformer_encoder_15204 transformer_encoder_15206 transformer_encoder_15208 transformer_encoder_15210 transformer_encoder_15212 transformer_encoder_15214 transformer_encoder_15216 transformer_encoder_15218 transformer_encoder_15220 transformer_encoder_15222 transformer_encoder_15224 transformer_encoder_15226 transformer_encoder_15228 transformer_encoder_15230 transformer_encoder_15232 transformer_encoder_15234 transformer_encoder_15236 transformer_encoder_15238 transformer_encoder_15240 transformer_encoder_15242 transformer_encoder_15244 transformer_encoder_15246 transformer_encoder_15248 transformer_encoder_15250 transformer_encoder_15252 transformer_encoder_15254 transformer_encoder_15256 transformer_encoder_15258 transformer_encoder_15260 transformer_encoder_15262 transformer_encoder_15264 transformer_encoder_15266 transformer_encoder_15268 transformer_encoder_15270 transformer_encoder_15272 transformer_encoder_15274 transformer_encoder_15276 transformer_encoder_15278 transformer_encoder_15280 transformer_encoder_15282 transformer_encoder_15284 transformer_encoder_15286 transformer_encoder_15288 transformer_encoder_15290 transformer_encoder_15292 transformer_encoder_15294 transformer_encoder_15296 transformer_encoder_15298 transformer_encoder_15300 transformer_encoder_15302 transformer_encoder_15304 transformer_encoder_15306 transformer_encoder_15308 transformer_encoder_15310 transformer_encoder_15312 transformer_encoder_15314 transformer_encoder_15316 transformer_encoder_15318 transformer_encoder_15320 transformer_encoder_15322 transformer_encoder_15324 transformer_encoder_15326 transformer_encoder_15328 transformer_encoder_15330 transformer_encoder_15332 transformer_encoder_15334 transformer_encoder_15336 transformer_encoder_15338 transformer_encoder_15340 transformer_encoder_15342 transformer_encoder_15344 transformer_encoder_15346 transformer_encoder_15348 transformer_encoder_15350 transformer_encoder_15352 transformer_encoder_15354 transformer_encoder_15356 transformer_encoder_15358 identity identity_1ˆ¢+transformer_encoder/StatefulPartitionedCall±1 +transformer_encoder/StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idstransformer_encoder_14962transformer_encoder_14964transformer_encoder_14966transformer_encoder_14968transformer_encoder_14970transformer_encoder_14972transformer_encoder_14974transformer_encoder_14976transformer_encoder_14978transformer_encoder_14980transformer_encoder_14982transformer_encoder_14984transformer_encoder_14986transformer_encoder_14988transformer_encoder_14990transformer_encoder_14992transformer_encoder_14994transformer_encoder_14996transformer_encoder_14998transformer_encoder_15000transformer_encoder_15002transformer_encoder_15004transformer_encoder_15006transformer_encoder_15008transformer_encoder_15010transformer_encoder_15012transformer_encoder_15014transformer_encoder_15016transformer_encoder_15018transformer_encoder_15020transformer_encoder_15022transformer_encoder_15024transformer_encoder_15026transformer_encoder_15028transformer_encoder_15030transformer_encoder_15032transformer_encoder_15034transformer_encoder_15036transformer_encoder_15038transformer_encoder_15040transformer_encoder_15042transformer_encoder_15044transformer_encoder_15046transformer_encoder_15048transformer_encoder_15050transformer_encoder_15052transformer_encoder_15054transformer_encoder_15056transformer_encoder_15058transformer_encoder_15060transformer_encoder_15062transformer_encoder_15064transformer_encoder_15066transformer_encoder_15068transformer_encoder_15070transformer_encoder_15072transformer_encoder_15074transformer_encoder_15076transformer_encoder_15078transformer_encoder_15080transformer_encoder_15082transformer_encoder_15084transformer_encoder_15086transformer_encoder_15088transformer_encoder_15090transformer_encoder_15092transformer_encoder_15094transformer_encoder_15096transformer_encoder_15098transformer_encoder_15100transformer_encoder_15102transformer_encoder_15104transformer_encoder_15106transformer_encoder_15108transformer_encoder_15110transformer_encoder_15112transformer_encoder_15114transformer_encoder_15116transformer_encoder_15118transformer_encoder_15120transformer_encoder_15122transformer_encoder_15124transformer_encoder_15126transformer_encoder_15128transformer_encoder_15130transformer_encoder_15132transformer_encoder_15134transformer_encoder_15136transformer_encoder_15138transformer_encoder_15140transformer_encoder_15142transformer_encoder_15144transformer_encoder_15146transformer_encoder_15148transformer_encoder_15150transformer_encoder_15152transformer_encoder_15154transformer_encoder_15156transformer_encoder_15158transformer_encoder_15160transformer_encoder_15162transformer_encoder_15164transformer_encoder_15166transformer_encoder_15168transformer_encoder_15170transformer_encoder_15172transformer_encoder_15174transformer_encoder_15176transformer_encoder_15178transformer_encoder_15180transformer_encoder_15182transformer_encoder_15184transformer_encoder_15186transformer_encoder_15188transformer_encoder_15190transformer_encoder_15192transformer_encoder_15194transformer_encoder_15196transformer_encoder_15198transformer_encoder_15200transformer_encoder_15202transformer_encoder_15204transformer_encoder_15206transformer_encoder_15208transformer_encoder_15210transformer_encoder_15212transformer_encoder_15214transformer_encoder_15216transformer_encoder_15218transformer_encoder_15220transformer_encoder_15222transformer_encoder_15224transformer_encoder_15226transformer_encoder_15228transformer_encoder_15230transformer_encoder_15232transformer_encoder_15234transformer_encoder_15236transformer_encoder_15238transformer_encoder_15240transformer_encoder_15242transformer_encoder_15244transformer_encoder_15246transformer_encoder_15248transformer_encoder_15250transformer_encoder_15252transformer_encoder_15254transformer_encoder_15256transformer_encoder_15258transformer_encoder_15260transformer_encoder_15262transformer_encoder_15264transformer_encoder_15266transformer_encoder_15268transformer_encoder_15270transformer_encoder_15272transformer_encoder_15274transformer_encoder_15276transformer_encoder_15278transformer_encoder_15280transformer_encoder_15282transformer_encoder_15284transformer_encoder_15286transformer_encoder_15288transformer_encoder_15290transformer_encoder_15292transformer_encoder_15294transformer_encoder_15296transformer_encoder_15298transformer_encoder_15300transformer_encoder_15302transformer_encoder_15304transformer_encoder_15306transformer_encoder_15308transformer_encoder_15310transformer_encoder_15312transformer_encoder_15314transformer_encoder_15316transformer_encoder_15318transformer_encoder_15320transformer_encoder_15322transformer_encoder_15324transformer_encoder_15326transformer_encoder_15328transformer_encoder_15330transformer_encoder_15332transformer_encoder_15334transformer_encoder_15336transformer_encoder_15338transformer_encoder_15340transformer_encoder_15342transformer_encoder_15344transformer_encoder_15346transformer_encoder_15348transformer_encoder_15350transformer_encoder_15352transformer_encoder_15354transformer_encoder_15356transformer_encoder_15358*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_129062- +transformer_encoder/StatefulPartitionedCall· IdentityIdentity4transformer_encoder/StatefulPartitionedCall:output:1,^transformer_encoder/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 IdentityČ Identity_1Identity4transformer_encoder/StatefulPartitionedCall:output:0,^transformer_encoder/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2Z +transformer_encoder/StatefulPartitionedCall+transformer_encoder/StatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ‰  é1 G__inference_functional_1_layer_call_and_return_conditional_losses_15771 input_word_ids input_mask input_type_ids transformer_encoder_15369 transformer_encoder_15371 transformer_encoder_15373 transformer_encoder_15375 transformer_encoder_15377 transformer_encoder_15379 transformer_encoder_15381 transformer_encoder_15383 transformer_encoder_15385 transformer_encoder_15387 transformer_encoder_15389 transformer_encoder_15391 transformer_encoder_15393 transformer_encoder_15395 transformer_encoder_15397 transformer_encoder_15399 transformer_encoder_15401 transformer_encoder_15403 transformer_encoder_15405 transformer_encoder_15407 transformer_encoder_15409 transformer_encoder_15411 transformer_encoder_15413 transformer_encoder_15415 transformer_encoder_15417 transformer_encoder_15419 transformer_encoder_15421 transformer_encoder_15423 transformer_encoder_15425 transformer_encoder_15427 transformer_encoder_15429 transformer_encoder_15431 transformer_encoder_15433 transformer_encoder_15435 transformer_encoder_15437 transformer_encoder_15439 transformer_encoder_15441 transformer_encoder_15443 transformer_encoder_15445 transformer_encoder_15447 transformer_encoder_15449 transformer_encoder_15451 transformer_encoder_15453 transformer_encoder_15455 transformer_encoder_15457 transformer_encoder_15459 transformer_encoder_15461 transformer_encoder_15463 transformer_encoder_15465 transformer_encoder_15467 transformer_encoder_15469 transformer_encoder_15471 transformer_encoder_15473 transformer_encoder_15475 transformer_encoder_15477 transformer_encoder_15479 transformer_encoder_15481 transformer_encoder_15483 transformer_encoder_15485 transformer_encoder_15487 transformer_encoder_15489 transformer_encoder_15491 transformer_encoder_15493 transformer_encoder_15495 transformer_encoder_15497 transformer_encoder_15499 transformer_encoder_15501 transformer_encoder_15503 transformer_encoder_15505 transformer_encoder_15507 transformer_encoder_15509 transformer_encoder_15511 transformer_encoder_15513 transformer_encoder_15515 transformer_encoder_15517 transformer_encoder_15519 transformer_encoder_15521 transformer_encoder_15523 transformer_encoder_15525 transformer_encoder_15527 transformer_encoder_15529 transformer_encoder_15531 transformer_encoder_15533 transformer_encoder_15535 transformer_encoder_15537 transformer_encoder_15539 transformer_encoder_15541 transformer_encoder_15543 transformer_encoder_15545 transformer_encoder_15547 transformer_encoder_15549 transformer_encoder_15551 transformer_encoder_15553 transformer_encoder_15555 transformer_encoder_15557 transformer_encoder_15559 transformer_encoder_15561 transformer_encoder_15563 transformer_encoder_15565 transformer_encoder_15567 transformer_encoder_15569 transformer_encoder_15571 transformer_encoder_15573 transformer_encoder_15575 transformer_encoder_15577 transformer_encoder_15579 transformer_encoder_15581 transformer_encoder_15583 transformer_encoder_15585 transformer_encoder_15587 transformer_encoder_15589 transformer_encoder_15591 transformer_encoder_15593 transformer_encoder_15595 transformer_encoder_15597 transformer_encoder_15599 transformer_encoder_15601 transformer_encoder_15603 transformer_encoder_15605 transformer_encoder_15607 transformer_encoder_15609 transformer_encoder_15611 transformer_encoder_15613 transformer_encoder_15615 transformer_encoder_15617 transformer_encoder_15619 transformer_encoder_15621 transformer_encoder_15623 transformer_encoder_15625 transformer_encoder_15627 transformer_encoder_15629 transformer_encoder_15631 transformer_encoder_15633 transformer_encoder_15635 transformer_encoder_15637 transformer_encoder_15639 transformer_encoder_15641 transformer_encoder_15643 transformer_encoder_15645 transformer_encoder_15647 transformer_encoder_15649 transformer_encoder_15651 transformer_encoder_15653 transformer_encoder_15655 transformer_encoder_15657 transformer_encoder_15659 transformer_encoder_15661 transformer_encoder_15663 transformer_encoder_15665 transformer_encoder_15667 transformer_encoder_15669 transformer_encoder_15671 transformer_encoder_15673 transformer_encoder_15675 transformer_encoder_15677 transformer_encoder_15679 transformer_encoder_15681 transformer_encoder_15683 transformer_encoder_15685 transformer_encoder_15687 transformer_encoder_15689 transformer_encoder_15691 transformer_encoder_15693 transformer_encoder_15695 transformer_encoder_15697 transformer_encoder_15699 transformer_encoder_15701 transformer_encoder_15703 transformer_encoder_15705 transformer_encoder_15707 transformer_encoder_15709 transformer_encoder_15711 transformer_encoder_15713 transformer_encoder_15715 transformer_encoder_15717 transformer_encoder_15719 transformer_encoder_15721 transformer_encoder_15723 transformer_encoder_15725 transformer_encoder_15727 transformer_encoder_15729 transformer_encoder_15731 transformer_encoder_15733 transformer_encoder_15735 transformer_encoder_15737 transformer_encoder_15739 transformer_encoder_15741 transformer_encoder_15743 transformer_encoder_15745 transformer_encoder_15747 transformer_encoder_15749 transformer_encoder_15751 transformer_encoder_15753 transformer_encoder_15755 transformer_encoder_15757 transformer_encoder_15759 transformer_encoder_15761 transformer_encoder_15763 transformer_encoder_15765 identity identity_1ˆ¢+transformer_encoder/StatefulPartitionedCall±1 +transformer_encoder/StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idstransformer_encoder_15369transformer_encoder_15371transformer_encoder_15373transformer_encoder_15375transformer_encoder_15377transformer_encoder_15379transformer_encoder_15381transformer_encoder_15383transformer_encoder_15385transformer_encoder_15387transformer_encoder_15389transformer_encoder_15391transformer_encoder_15393transformer_encoder_15395transformer_encoder_15397transformer_encoder_15399transformer_encoder_15401transformer_encoder_15403transformer_encoder_15405transformer_encoder_15407transformer_encoder_15409transformer_encoder_15411transformer_encoder_15413transformer_encoder_15415transformer_encoder_15417transformer_encoder_15419transformer_encoder_15421transformer_encoder_15423transformer_encoder_15425transformer_encoder_15427transformer_encoder_15429transformer_encoder_15431transformer_encoder_15433transformer_encoder_15435transformer_encoder_15437transformer_encoder_15439transformer_encoder_15441transformer_encoder_15443transformer_encoder_15445transformer_encoder_15447transformer_encoder_15449transformer_encoder_15451transformer_encoder_15453transformer_encoder_15455transformer_encoder_15457transformer_encoder_15459transformer_encoder_15461transformer_encoder_15463transformer_encoder_15465transformer_encoder_15467transformer_encoder_15469transformer_encoder_15471transformer_encoder_15473transformer_encoder_15475transformer_encoder_15477transformer_encoder_15479transformer_encoder_15481transformer_encoder_15483transformer_encoder_15485transformer_encoder_15487transformer_encoder_15489transformer_encoder_15491transformer_encoder_15493transformer_encoder_15495transformer_encoder_15497transformer_encoder_15499transformer_encoder_15501transformer_encoder_15503transformer_encoder_15505transformer_encoder_15507transformer_encoder_15509transformer_encoder_15511transformer_encoder_15513transformer_encoder_15515transformer_encoder_15517transformer_encoder_15519transformer_encoder_15521transformer_encoder_15523transformer_encoder_15525transformer_encoder_15527transformer_encoder_15529transformer_encoder_15531transformer_encoder_15533transformer_encoder_15535transformer_encoder_15537transformer_encoder_15539transformer_encoder_15541transformer_encoder_15543transformer_encoder_15545transformer_encoder_15547transformer_encoder_15549transformer_encoder_15551transformer_encoder_15553transformer_encoder_15555transformer_encoder_15557transformer_encoder_15559transformer_encoder_15561transformer_encoder_15563transformer_encoder_15565transformer_encoder_15567transformer_encoder_15569transformer_encoder_15571transformer_encoder_15573transformer_encoder_15575transformer_encoder_15577transformer_encoder_15579transformer_encoder_15581transformer_encoder_15583transformer_encoder_15585transformer_encoder_15587transformer_encoder_15589transformer_encoder_15591transformer_encoder_15593transformer_encoder_15595transformer_encoder_15597transformer_encoder_15599transformer_encoder_15601transformer_encoder_15603transformer_encoder_15605transformer_encoder_15607transformer_encoder_15609transformer_encoder_15611transformer_encoder_15613transformer_encoder_15615transformer_encoder_15617transformer_encoder_15619transformer_encoder_15621transformer_encoder_15623transformer_encoder_15625transformer_encoder_15627transformer_encoder_15629transformer_encoder_15631transformer_encoder_15633transformer_encoder_15635transformer_encoder_15637transformer_encoder_15639transformer_encoder_15641transformer_encoder_15643transformer_encoder_15645transformer_encoder_15647transformer_encoder_15649transformer_encoder_15651transformer_encoder_15653transformer_encoder_15655transformer_encoder_15657transformer_encoder_15659transformer_encoder_15661transformer_encoder_15663transformer_encoder_15665transformer_encoder_15667transformer_encoder_15669transformer_encoder_15671transformer_encoder_15673transformer_encoder_15675transformer_encoder_15677transformer_encoder_15679transformer_encoder_15681transformer_encoder_15683transformer_encoder_15685transformer_encoder_15687transformer_encoder_15689transformer_encoder_15691transformer_encoder_15693transformer_encoder_15695transformer_encoder_15697transformer_encoder_15699transformer_encoder_15701transformer_encoder_15703transformer_encoder_15705transformer_encoder_15707transformer_encoder_15709transformer_encoder_15711transformer_encoder_15713transformer_encoder_15715transformer_encoder_15717transformer_encoder_15719transformer_encoder_15721transformer_encoder_15723transformer_encoder_15725transformer_encoder_15727transformer_encoder_15729transformer_encoder_15731transformer_encoder_15733transformer_encoder_15735transformer_encoder_15737transformer_encoder_15739transformer_encoder_15741transformer_encoder_15743transformer_encoder_15745transformer_encoder_15747transformer_encoder_15749transformer_encoder_15751transformer_encoder_15753transformer_encoder_15755transformer_encoder_15757transformer_encoder_15759transformer_encoder_15761transformer_encoder_15763transformer_encoder_15765*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_137392- +transformer_encoder/StatefulPartitionedCall· IdentityIdentity4transformer_encoder/StatefulPartitionedCall:output:1,^transformer_encoder/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 IdentityČ Identity_1Identity4transformer_encoder/StatefulPartitionedCall:output:0,^transformer_encoder/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2Z +transformer_encoder/StatefulPartitionedCall+transformer_encoder/StatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ą Ż7 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12475 input_word_ids input_mask input_type_ids word_embeddings_12054 position_embedding_12057 type_embeddings_12060 embeddings_layer_norm_12064 embeddings_layer_norm_12066 transformer_layer_0_12071 transformer_layer_0_12073 transformer_layer_0_12075 transformer_layer_0_12077 transformer_layer_0_12079 transformer_layer_0_12081 transformer_layer_0_12083 transformer_layer_0_12085 transformer_layer_0_12087 transformer_layer_0_12089 transformer_layer_0_12091 transformer_layer_0_12093 transformer_layer_0_12095 transformer_layer_0_12097 transformer_layer_0_12099 transformer_layer_0_12101 transformer_layer_1_12104 transformer_layer_1_12106 transformer_layer_1_12108 transformer_layer_1_12110 transformer_layer_1_12112 transformer_layer_1_12114 transformer_layer_1_12116 transformer_layer_1_12118 transformer_layer_1_12120 transformer_layer_1_12122 transformer_layer_1_12124 transformer_layer_1_12126 transformer_layer_1_12128 transformer_layer_1_12130 transformer_layer_1_12132 transformer_layer_1_12134 transformer_layer_2_12137 transformer_layer_2_12139 transformer_layer_2_12141 transformer_layer_2_12143 transformer_layer_2_12145 transformer_layer_2_12147 transformer_layer_2_12149 transformer_layer_2_12151 transformer_layer_2_12153 transformer_layer_2_12155 transformer_layer_2_12157 transformer_layer_2_12159 transformer_layer_2_12161 transformer_layer_2_12163 transformer_layer_2_12165 transformer_layer_2_12167 transformer_layer_3_12170 transformer_layer_3_12172 transformer_layer_3_12174 transformer_layer_3_12176 transformer_layer_3_12178 transformer_layer_3_12180 transformer_layer_3_12182 transformer_layer_3_12184 transformer_layer_3_12186 transformer_layer_3_12188 transformer_layer_3_12190 transformer_layer_3_12192 transformer_layer_3_12194 transformer_layer_3_12196 transformer_layer_3_12198 transformer_layer_3_12200 transformer_layer_4_12203 transformer_layer_4_12205 transformer_layer_4_12207 transformer_layer_4_12209 transformer_layer_4_12211 transformer_layer_4_12213 transformer_layer_4_12215 transformer_layer_4_12217 transformer_layer_4_12219 transformer_layer_4_12221 transformer_layer_4_12223 transformer_layer_4_12225 transformer_layer_4_12227 transformer_layer_4_12229 transformer_layer_4_12231 transformer_layer_4_12233 transformer_layer_5_12236 transformer_layer_5_12238 transformer_layer_5_12240 transformer_layer_5_12242 transformer_layer_5_12244 transformer_layer_5_12246 transformer_layer_5_12248 transformer_layer_5_12250 transformer_layer_5_12252 transformer_layer_5_12254 transformer_layer_5_12256 transformer_layer_5_12258 transformer_layer_5_12260 transformer_layer_5_12262 transformer_layer_5_12264 transformer_layer_5_12266 transformer_layer_6_12269 transformer_layer_6_12271 transformer_layer_6_12273 transformer_layer_6_12275 transformer_layer_6_12277 transformer_layer_6_12279 transformer_layer_6_12281 transformer_layer_6_12283 transformer_layer_6_12285 transformer_layer_6_12287 transformer_layer_6_12289 transformer_layer_6_12291 transformer_layer_6_12293 transformer_layer_6_12295 transformer_layer_6_12297 transformer_layer_6_12299 transformer_layer_7_12302 transformer_layer_7_12304 transformer_layer_7_12306 transformer_layer_7_12308 transformer_layer_7_12310 transformer_layer_7_12312 transformer_layer_7_12314 transformer_layer_7_12316 transformer_layer_7_12318 transformer_layer_7_12320 transformer_layer_7_12322 transformer_layer_7_12324 transformer_layer_7_12326 transformer_layer_7_12328 transformer_layer_7_12330 transformer_layer_7_12332 transformer_layer_8_12335 transformer_layer_8_12337 transformer_layer_8_12339 transformer_layer_8_12341 transformer_layer_8_12343 transformer_layer_8_12345 transformer_layer_8_12347 transformer_layer_8_12349 transformer_layer_8_12351 transformer_layer_8_12353 transformer_layer_8_12355 transformer_layer_8_12357 transformer_layer_8_12359 transformer_layer_8_12361 transformer_layer_8_12363 transformer_layer_8_12365 transformer_layer_9_12368 transformer_layer_9_12370 transformer_layer_9_12372 transformer_layer_9_12374 transformer_layer_9_12376 transformer_layer_9_12378 transformer_layer_9_12380 transformer_layer_9_12382 transformer_layer_9_12384 transformer_layer_9_12386 transformer_layer_9_12388 transformer_layer_9_12390 transformer_layer_9_12392 transformer_layer_9_12394 transformer_layer_9_12396 transformer_layer_9_12398 transformer_layer_10_12401 transformer_layer_10_12403 transformer_layer_10_12405 transformer_layer_10_12407 transformer_layer_10_12409 transformer_layer_10_12411 transformer_layer_10_12413 transformer_layer_10_12415 transformer_layer_10_12417 transformer_layer_10_12419 transformer_layer_10_12421 transformer_layer_10_12423 transformer_layer_10_12425 transformer_layer_10_12427 transformer_layer_10_12429 transformer_layer_10_12431 transformer_layer_11_12434 transformer_layer_11_12436 transformer_layer_11_12438 transformer_layer_11_12440 transformer_layer_11_12442 transformer_layer_11_12444 transformer_layer_11_12446 transformer_layer_11_12448 transformer_layer_11_12450 transformer_layer_11_12452 transformer_layer_11_12454 transformer_layer_11_12456 transformer_layer_11_12458 transformer_layer_11_12460 transformer_layer_11_12462 transformer_layer_11_12464 pooler_transform_12468 pooler_transform_12470 identity identity_1ˆ¢-embeddings/layer_norm/StatefulPartitionedCall¢(pooler_transform/StatefulPartitionedCall¢*position_embedding/StatefulPartitionedCall¢+transformer/layer_0/StatefulPartitionedCall¢+transformer/layer_1/StatefulPartitionedCall¢,transformer/layer_10/StatefulPartitionedCall¢,transformer/layer_11/StatefulPartitionedCall¢+transformer/layer_2/StatefulPartitionedCall¢+transformer/layer_3/StatefulPartitionedCall¢+transformer/layer_4/StatefulPartitionedCall¢+transformer/layer_5/StatefulPartitionedCall¢+transformer/layer_6/StatefulPartitionedCall¢+transformer/layer_7/StatefulPartitionedCall¢+transformer/layer_8/StatefulPartitionedCall¢+transformer/layer_9/StatefulPartitionedCall¢'type_embeddings/StatefulPartitionedCall¢'word_embeddings/StatefulPartitionedCall­ 'word_embeddings/StatefulPartitionedCallStatefulPartitionedCallinput_word_idsword_embeddings_12054* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_word_embeddings_layer_call_and_return_conditional_losses_76282) 'word_embeddings/StatefulPartitionedCallŪ *position_embedding/StatefulPartitionedCallStatefulPartitionedCall0word_embeddings/StatefulPartitionedCall:output:0position_embedding_12057* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*U fPRN L__inference_position_embedding_layer_call_and_return_conditional_losses_76632, *position_embedding/StatefulPartitionedCall­ 'type_embeddings/StatefulPartitionedCallStatefulPartitionedCallinput_type_idstype_embeddings_12060* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_type_embeddings_layer_call_and_return_conditional_losses_76932) 'type_embeddings/StatefulPartitionedCallį add/PartitionedCallPartitionedCall0word_embeddings/StatefulPartitionedCall:output:03position_embedding/StatefulPartitionedCall:output:00type_embeddings/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*F fAR? =__inference_add_layer_call_and_return_conditional_losses_77132 add/PartitionedCallņ -embeddings/layer_norm/StatefulPartitionedCallStatefulPartitionedCalladd/PartitionedCall:output:0embeddings_layer_norm_12064embeddings_layer_norm_12066* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*X fSRQ O__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_77452/ -embeddings/layer_norm/StatefulPartitionedCallŠ dropout/PartitionedCallPartitionedCall6embeddings/layer_norm/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_dropout_layer_call_and_return_conditional_losses_77782 dropout/PartitionedCall­ #self_attention_mask/PartitionedCallPartitionedCall dropout/PartitionedCall:output:0 input_mask* Tin 2* Tout 2* _collective_manager_ids *= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*V fQRO M__inference_self_attention_mask_layer_call_and_return_conditional_losses_78282% #self_attention_mask/PartitionedCall± +transformer/layer_0/StatefulPartitionedCallStatefulPartitionedCall dropout/PartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_0_12071transformer_layer_0_12073transformer_layer_0_12075transformer_layer_0_12077transformer_layer_0_12079transformer_layer_0_12081transformer_layer_0_12083transformer_layer_0_12085transformer_layer_0_12087transformer_layer_0_12089transformer_layer_0_12091transformer_layer_0_12093transformer_layer_0_12095transformer_layer_0_12097transformer_layer_0_12099transformer_layer_0_12101* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_80722- +transformer/layer_0/StatefulPartitionedCallÅ +transformer/layer_1/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_0/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_1_12104transformer_layer_1_12106transformer_layer_1_12108transformer_layer_1_12110transformer_layer_1_12112transformer_layer_1_12114transformer_layer_1_12116transformer_layer_1_12118transformer_layer_1_12120transformer_layer_1_12122transformer_layer_1_12124transformer_layer_1_12126transformer_layer_1_12128transformer_layer_1_12130transformer_layer_1_12132transformer_layer_1_12134* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_84182- +transformer/layer_1/StatefulPartitionedCallÅ +transformer/layer_2/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_1/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_2_12137transformer_layer_2_12139transformer_layer_2_12141transformer_layer_2_12143transformer_layer_2_12145transformer_layer_2_12147transformer_layer_2_12149transformer_layer_2_12151transformer_layer_2_12153transformer_layer_2_12155transformer_layer_2_12157transformer_layer_2_12159transformer_layer_2_12161transformer_layer_2_12163transformer_layer_2_12165transformer_layer_2_12167* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_87642- +transformer/layer_2/StatefulPartitionedCallÅ +transformer/layer_3/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_2/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_3_12170transformer_layer_3_12172transformer_layer_3_12174transformer_layer_3_12176transformer_layer_3_12178transformer_layer_3_12180transformer_layer_3_12182transformer_layer_3_12184transformer_layer_3_12186transformer_layer_3_12188transformer_layer_3_12190transformer_layer_3_12192transformer_layer_3_12194transformer_layer_3_12196transformer_layer_3_12198transformer_layer_3_12200* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_91102- +transformer/layer_3/StatefulPartitionedCallÅ +transformer/layer_4/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_3/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_4_12203transformer_layer_4_12205transformer_layer_4_12207transformer_layer_4_12209transformer_layer_4_12211transformer_layer_4_12213transformer_layer_4_12215transformer_layer_4_12217transformer_layer_4_12219transformer_layer_4_12221transformer_layer_4_12223transformer_layer_4_12225transformer_layer_4_12227transformer_layer_4_12229transformer_layer_4_12231transformer_layer_4_12233* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_94562- +transformer/layer_4/StatefulPartitionedCallÅ +transformer/layer_5/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_4/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_5_12236transformer_layer_5_12238transformer_layer_5_12240transformer_layer_5_12242transformer_layer_5_12244transformer_layer_5_12246transformer_layer_5_12248transformer_layer_5_12250transformer_layer_5_12252transformer_layer_5_12254transformer_layer_5_12256transformer_layer_5_12258transformer_layer_5_12260transformer_layer_5_12262transformer_layer_5_12264transformer_layer_5_12266* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_98022- +transformer/layer_5/StatefulPartitionedCallĘ +transformer/layer_6/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_5/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_6_12269transformer_layer_6_12271transformer_layer_6_12273transformer_layer_6_12275transformer_layer_6_12277transformer_layer_6_12279transformer_layer_6_12281transformer_layer_6_12283transformer_layer_6_12285transformer_layer_6_12287transformer_layer_6_12289transformer_layer_6_12291transformer_layer_6_12293transformer_layer_6_12295transformer_layer_6_12297transformer_layer_6_12299* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_101482- +transformer/layer_6/StatefulPartitionedCallĘ +transformer/layer_7/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_6/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_7_12302transformer_layer_7_12304transformer_layer_7_12306transformer_layer_7_12308transformer_layer_7_12310transformer_layer_7_12312transformer_layer_7_12314transformer_layer_7_12316transformer_layer_7_12318transformer_layer_7_12320transformer_layer_7_12322transformer_layer_7_12324transformer_layer_7_12326transformer_layer_7_12328transformer_layer_7_12330transformer_layer_7_12332* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_104942- +transformer/layer_7/StatefulPartitionedCallĘ +transformer/layer_8/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_7/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_8_12335transformer_layer_8_12337transformer_layer_8_12339transformer_layer_8_12341transformer_layer_8_12343transformer_layer_8_12345transformer_layer_8_12347transformer_layer_8_12349transformer_layer_8_12351transformer_layer_8_12353transformer_layer_8_12355transformer_layer_8_12357transformer_layer_8_12359transformer_layer_8_12361transformer_layer_8_12363transformer_layer_8_12365* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_108402- +transformer/layer_8/StatefulPartitionedCallĘ +transformer/layer_9/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_8/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_9_12368transformer_layer_9_12370transformer_layer_9_12372transformer_layer_9_12374transformer_layer_9_12376transformer_layer_9_12378transformer_layer_9_12380transformer_layer_9_12382transformer_layer_9_12384transformer_layer_9_12386transformer_layer_9_12388transformer_layer_9_12390transformer_layer_9_12392transformer_layer_9_12394transformer_layer_9_12396transformer_layer_9_12398* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_111862- +transformer/layer_9/StatefulPartitionedCallŁ ,transformer/layer_10/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_9/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_10_12401transformer_layer_10_12403transformer_layer_10_12405transformer_layer_10_12407transformer_layer_10_12409transformer_layer_10_12411transformer_layer_10_12413transformer_layer_10_12415transformer_layer_10_12417transformer_layer_10_12419transformer_layer_10_12421transformer_layer_10_12423transformer_layer_10_12425transformer_layer_10_12427transformer_layer_10_12429transformer_layer_10_12431* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_115322. ,transformer/layer_10/StatefulPartitionedCallŚ ,transformer/layer_11/StatefulPartitionedCallStatefulPartitionedCall5transformer/layer_10/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_11_12434transformer_layer_11_12436transformer_layer_11_12438transformer_layer_11_12440transformer_layer_11_12442transformer_layer_11_12444transformer_layer_11_12446transformer_layer_11_12448transformer_layer_11_12450transformer_layer_11_12452transformer_layer_11_12454transformer_layer_11_12456transformer_layer_11_12458transformer_layer_11_12460transformer_layer_11_12462transformer_layer_11_12464* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_118782. ,transformer/layer_11/StatefulPartitionedCallś lambda/PartitionedCallPartitionedCall5transformer/layer_11/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_lambda_layer_call_and_return_conditional_losses_120072 lambda/PartitionedCallŠ (pooler_transform/StatefulPartitionedCallStatefulPartitionedCalllambda/PartitionedCall:output:0pooler_transform_12468pooler_transform_12470* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*T fORM K__inference_pooler_transform_layer_call_and_return_conditional_losses_120312* (pooler_transform/StatefulPartitionedCall IdentityIdentity5transformer/layer_11/StatefulPartitionedCall:output:0.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity Identity_1Identity1pooler_transform/StatefulPartitionedCall:output:0.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2^ -embeddings/layer_norm/StatefulPartitionedCall-embeddings/layer_norm/StatefulPartitionedCall2T (pooler_transform/StatefulPartitionedCall(pooler_transform/StatefulPartitionedCall2X *position_embedding/StatefulPartitionedCall*position_embedding/StatefulPartitionedCall2Z +transformer/layer_0/StatefulPartitionedCall+transformer/layer_0/StatefulPartitionedCall2Z +transformer/layer_1/StatefulPartitionedCall+transformer/layer_1/StatefulPartitionedCall2\ ,transformer/layer_10/StatefulPartitionedCall,transformer/layer_10/StatefulPartitionedCall2\ ,transformer/layer_11/StatefulPartitionedCall,transformer/layer_11/StatefulPartitionedCall2Z +transformer/layer_2/StatefulPartitionedCall+transformer/layer_2/StatefulPartitionedCall2Z +transformer/layer_3/StatefulPartitionedCall+transformer/layer_3/StatefulPartitionedCall2Z +transformer/layer_4/StatefulPartitionedCall+transformer/layer_4/StatefulPartitionedCall2Z +transformer/layer_5/StatefulPartitionedCall+transformer/layer_5/StatefulPartitionedCall2Z +transformer/layer_6/StatefulPartitionedCall+transformer/layer_6/StatefulPartitionedCall2Z +transformer/layer_7/StatefulPartitionedCall+transformer/layer_7/StatefulPartitionedCall2Z +transformer/layer_8/StatefulPartitionedCall+transformer/layer_8/StatefulPartitionedCall2Z +transformer/layer_9/StatefulPartitionedCall+transformer/layer_9/StatefulPartitionedCall2R 'type_embeddings/StatefulPartitionedCall'type_embeddings/StatefulPartitionedCall2R 'word_embeddings/StatefulPartitionedCall'word_embeddings/StatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : Żį ś˜ G__inference_functional_1_layer_call_and_return_conditional_losses_20746 inputs_0 inputs_1 inputs_27 3transformer_encoder_word_embeddings_gather_resourceB >transformer_encoder_position_embedding_readvariableop_resourceF Btransformer_encoder_type_embeddings_matmul_readvariableop_resourceS Otransformer_encoder_embeddings_layer_norm_batchnorm_mul_readvariableop_resourceO Ktransformer_encoder_embeddings_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_0_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_0_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_0_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_0_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_0_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_0_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_0_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_0_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_0_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_1_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_1_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_1_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_1_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_1_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_1_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_1_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_1_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_2_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_2_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_2_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_2_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_2_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_2_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_2_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_2_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_3_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_3_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_3_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_3_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_3_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_3_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_3_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_3_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_4_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_4_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_4_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_4_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_4_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_4_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_4_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_4_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_5_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_5_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_5_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_5_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_5_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_5_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_5_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_5_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_6_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_6_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_6_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_6_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_6_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_6_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_6_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_6_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_7_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_7_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_7_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_7_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_7_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_7_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_7_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_7_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_8_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_8_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_8_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_8_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_8_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_8_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_8_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_8_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_9_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_9_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_9_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_9_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_9_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_9_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_9_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_9_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_readvariableop_resourceg ctransformer_encoder_transformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_10_self_attention_query_add_readvariableop_resourcee atransformer_encoder_transformer_layer_10_self_attention_key_einsum_einsum_readvariableop_resource[ Wtransformer_encoder_transformer_layer_10_self_attention_key_add_readvariableop_resourceg ctransformer_encoder_transformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_10_self_attention_value_add_readvariableop_resourcer ntransformer_encoder_transformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resourceh dtransformer_encoder_transformer_layer_10_self_attention_attention_output_add_readvariableop_resourcel htransformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceh dtransformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resource_ [transformer_encoder_transformer_layer_10_intermediate_einsum_einsum_readvariableop_resourceU Qtransformer_encoder_transformer_layer_10_intermediate_add_readvariableop_resourceY Utransformer_encoder_transformer_layer_10_output_einsum_einsum_readvariableop_resourceO Ktransformer_encoder_transformer_layer_10_output_add_readvariableop_resourced `transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_mul_readvariableop_resource` \transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_readvariableop_resourceg ctransformer_encoder_transformer_layer_11_self_attention_query_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_11_self_attention_query_add_readvariableop_resourcee atransformer_encoder_transformer_layer_11_self_attention_key_einsum_einsum_readvariableop_resource[ Wtransformer_encoder_transformer_layer_11_self_attention_key_add_readvariableop_resourceg ctransformer_encoder_transformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_11_self_attention_value_add_readvariableop_resourcer ntransformer_encoder_transformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resourceh dtransformer_encoder_transformer_layer_11_self_attention_attention_output_add_readvariableop_resourcel htransformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceh dtransformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resource_ [transformer_encoder_transformer_layer_11_intermediate_einsum_einsum_readvariableop_resourceU Qtransformer_encoder_transformer_layer_11_intermediate_add_readvariableop_resourceY Utransformer_encoder_transformer_layer_11_output_einsum_einsum_readvariableop_resourceO Ktransformer_encoder_transformer_layer_11_output_add_readvariableop_resourced `transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_mul_readvariableop_resource` \transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_readvariableop_resourceG Ctransformer_encoder_pooler_transform_matmul_readvariableop_resourceH Dtransformer_encoder_pooler_transform_biasadd_readvariableop_resource identity identity_1ˆ¹ 1transformer_encoder/word_embeddings/Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’23 1transformer_encoder/word_embeddings/Reshape/shapeŁ +transformer_encoder/word_embeddings/ReshapeReshapeinputs_0:transformer_encoder/word_embeddings/Reshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2- +transformer_encoder/word_embeddings/Reshapeœ *transformer_encoder/word_embeddings/GatherResourceGather3transformer_encoder_word_embeddings_gather_resource4transformer_encoder/word_embeddings/Reshape:output:0* Tindices0*( _output_shapes : ’’’’’’’’’€* dtype02, *transformer_encoder/word_embeddings/GatherŠ ,transformer_encoder/word_embeddings/IdentityIdentity3transformer_encoder/word_embeddings/Gather:output:0* T0*( _output_shapes : ’’’’’’’’’€2. ,transformer_encoder/word_embeddings/IdentityŽ )transformer_encoder/word_embeddings/ShapeShapeinputs_0* T0* _output_shapes :2+ )transformer_encoder/word_embeddings/Shapeµ 3transformer_encoder/word_embeddings/concat/values_1Const* _output_shapes :* dtype0* valueB :€25 3transformer_encoder/word_embeddings/concat/values_1¤ /transformer_encoder/word_embeddings/concat/axisConst* _output_shapes :* dtype0* value B:21 /transformer_encoder/word_embeddings/concat/axis¾ *transformer_encoder/word_embeddings/concatConcatV22transformer_encoder/word_embeddings/Shape:output:0 transformer_encoder/position_embedding/strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2@ >transformer_encoder/position_embedding/strided_slice_1/stack_1Ź >transformer_encoder/position_embedding/strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2@ >transformer_encoder/position_embedding/strided_slice_1/stack_2Ö 6transformer_encoder/position_embedding/strided_slice_1 StridedSlice5transformer_encoder/position_embedding/Shape:output:0Etransformer_encoder/position_embedding/strided_slice_1/stack:output:0Gtransformer_encoder/position_embedding/strided_slice_1/stack_1:output:0Gtransformer_encoder/position_embedding/strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask28 6transformer_encoder/position_embedding/strided_slice_1ļ 5transformer_encoder/position_embedding/ReadVariableOpReadVariableOp>transformer_encoder_position_embedding_readvariableop_resource* _output_shapes : €€* dtype027 5transformer_encoder/position_embedding/ReadVariableOpĶ transformer_encoder/position_embedding/strided_slice_2/stack_1Pack?transformer_encoder/position_embedding/strided_slice_1:output:0Itransformer_encoder/position_embedding/strided_slice_2/stack_1/1:output:0* N* T0* _output_shapes :2@ >transformer_encoder/position_embedding/strided_slice_2/stack_1Ń >transformer_encoder/position_embedding/strided_slice_2/stack_2Const* _output_shapes :* dtype0* valueB"2@ >transformer_encoder/position_embedding/strided_slice_2/stack_2ś 6transformer_encoder/position_embedding/strided_slice_2 StridedSlice=transformer_encoder/position_embedding/ReadVariableOp:value:0Etransformer_encoder/position_embedding/strided_slice_2/stack:output:0Gtransformer_encoder/position_embedding/strided_slice_2/stack_1:output:0Gtransformer_encoder/position_embedding/strided_slice_2/stack_2:output:0* Index0* T0*( _output_shapes : ’’’’’’’’’€* begin_mask* end_mask28 6transformer_encoder/position_embedding/strided_slice_2» :transformer_encoder/position_embedding/BroadcastTo/shape/2Const* _output_shapes :* dtype0* value B:€2< :transformer_encoder/position_embedding/BroadcastTo/shape/2ļ 8transformer_encoder/position_embedding/BroadcastTo/shapePack=transformer_encoder/position_embedding/strided_slice:output:0?transformer_encoder/position_embedding/strided_slice_1:output:0Ctransformer_encoder/position_embedding/BroadcastTo/shape/2:output:0* N* T0* _output_shapes :2: 8transformer_encoder/position_embedding/BroadcastTo/shape» 2transformer_encoder/position_embedding/BroadcastTo BroadcastTo?transformer_encoder/position_embedding/strided_slice_2:output:0Atransformer_encoder/position_embedding/BroadcastTo/shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer_encoder/position_embedding/BroadcastTo¹ 1transformer_encoder/type_embeddings/Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’23 1transformer_encoder/type_embeddings/Reshape/shapeŁ +transformer_encoder/type_embeddings/ReshapeReshapeinputs_2:transformer_encoder/type_embeddings/Reshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2- +transformer_encoder/type_embeddings/Reshape± 4transformer_encoder/type_embeddings/one_hot/on_valueConst* _output_shapes :* dtype0* value B *€?26 4transformer_encoder/type_embeddings/one_hot/on_value³ 5transformer_encoder/type_embeddings/one_hot/off_valueConst* _output_shapes :* dtype0* value B *27 5transformer_encoder/type_embeddings/one_hot/off_valueØ 1transformer_encoder/type_embeddings/one_hot/depthConst* _output_shapes :* dtype0* value B:23 1transformer_encoder/type_embeddings/one_hot/depth‘ +transformer_encoder/type_embeddings/one_hotOneHot4transformer_encoder/type_embeddings/Reshape:output:0:transformer_encoder/type_embeddings/one_hot/depth:output:0=transformer_encoder/type_embeddings/one_hot/on_value:output:0>transformer_encoder/type_embeddings/one_hot/off_value:output:0* T0* TI0*' _output_shapes : ’’’’’’’’’2- +transformer_encoder/type_embeddings/one_hotś 9transformer_encoder/type_embeddings/MatMul/ReadVariableOpReadVariableOpBtransformer_encoder_type_embeddings_matmul_readvariableop_resource* _output_shapes : €* dtype02; 9transformer_encoder/type_embeddings/MatMul/ReadVariableOpŽ *transformer_encoder/type_embeddings/MatMulMatMul4transformer_encoder/type_embeddings/one_hot:output:0Atransformer_encoder/type_embeddings/MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2, *transformer_encoder/type_embeddings/MatMulŽ )transformer_encoder/type_embeddings/ShapeShapeinputs_2* T0* _output_shapes :2+ )transformer_encoder/type_embeddings/Shapeµ 3transformer_encoder/type_embeddings/concat/values_1Const* _output_shapes :* dtype0* valueB :€25 3transformer_encoder/type_embeddings/concat/values_1¤ /transformer_encoder/type_embeddings/concat/axisConst* _output_shapes :* dtype0* value B:21 /transformer_encoder/type_embeddings/concat/axis¾ *transformer_encoder/type_embeddings/concatConcatV22transformer_encoder/type_embeddings/Shape:output:0transformer_encoder/embeddings/layer_norm/moments/StopGradient StopGradient?transformer_encoder/embeddings/layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2@ >transformer_encoder/embeddings/layer_norm/moments/StopGradientĖ Ctransformer_encoder/embeddings/layer_norm/moments/SquaredDifferenceSquaredDifference!transformer_encoder/add/add_1:z:0Gtransformer_encoder/embeddings/layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2E Ctransformer_encoder/embeddings/layer_norm/moments/SquaredDifferenceę Ltransformer_encoder/embeddings/layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer_encoder/embeddings/layer_norm/moments/variance/reduction_indicesš :transformer_encoder/embeddings/layer_norm/moments/varianceMeanGtransformer_encoder/embeddings/layer_norm/moments/SquaredDifference:z:0Utransformer_encoder/embeddings/layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer_encoder/embeddings/layer_norm/moments/variance» 9transformer_encoder/embeddings/layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2; 9transformer_encoder/embeddings/layer_norm/batchnorm/add/yĆ 7transformer_encoder/embeddings/layer_norm/batchnorm/addAddV2Ctransformer_encoder/embeddings/layer_norm/moments/variance:output:0Btransformer_encoder/embeddings/layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’29 7transformer_encoder/embeddings/layer_norm/batchnorm/addū 9transformer_encoder/embeddings/layer_norm/batchnorm/RsqrtRsqrt;transformer_encoder/embeddings/layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2; 9transformer_encoder/embeddings/layer_norm/batchnorm/Rsqrt Ftransformer_encoder/embeddings/layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpOtransformer_encoder_embeddings_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer_encoder/embeddings/layer_norm/batchnorm/mul/ReadVariableOpČ 7transformer_encoder/embeddings/layer_norm/batchnorm/mulMul=transformer_encoder/embeddings/layer_norm/batchnorm/Rsqrt:y:0Ntransformer_encoder/embeddings/layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer_encoder/embeddings/layer_norm/batchnorm/mul 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_1Mul!transformer_encoder/add/add_1:z:0;transformer_encoder/embeddings/layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_1» 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_2Mul?transformer_encoder/embeddings/layer_norm/moments/mean:output:0;transformer_encoder/embeddings/layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_2‘ Btransformer_encoder/embeddings/layer_norm/batchnorm/ReadVariableOpReadVariableOpKtransformer_encoder_embeddings_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02D Btransformer_encoder/embeddings/layer_norm/batchnorm/ReadVariableOpÄ 7transformer_encoder/embeddings/layer_norm/batchnorm/subSubJtransformer_encoder/embeddings/layer_norm/batchnorm/ReadVariableOp:value:0=transformer_encoder/embeddings/layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer_encoder/embeddings/layer_norm/batchnorm/sub» 9transformer_encoder/embeddings/layer_norm/batchnorm/add_1AddV2=transformer_encoder/embeddings/layer_norm/batchnorm/mul_1:z:0;transformer_encoder/embeddings/layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/embeddings/layer_norm/batchnorm/add_1× $transformer_encoder/dropout/IdentityIdentity=transformer_encoder/embeddings/layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2& $transformer_encoder/dropout/Identity» -transformer_encoder/self_attention_mask/ShapeShape-transformer_encoder/dropout/Identity:output:0* T0* _output_shapes :2/ -transformer_encoder/self_attention_mask/ShapeÄ ;transformer_encoder/self_attention_mask/strided_slice/stackConst* _output_shapes :* dtype0* value B :2= ;transformer_encoder/self_attention_mask/strided_slice/stackČ =transformer_encoder/self_attention_mask/strided_slice/stack_1Const* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice/stack_1Č =transformer_encoder/self_attention_mask/strided_slice/stack_2Const* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice/stack_2Ņ 5transformer_encoder/self_attention_mask/strided_slice StridedSlice6transformer_encoder/self_attention_mask/Shape:output:0Dtransformer_encoder/self_attention_mask/strided_slice/stack:output:0Ftransformer_encoder/self_attention_mask/strided_slice/stack_1:output:0Ftransformer_encoder/self_attention_mask/strided_slice/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask27 5transformer_encoder/self_attention_mask/strided_sliceČ =transformer_encoder/self_attention_mask/strided_slice_1/stackConst* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice_1/stackĢ ?transformer_encoder/self_attention_mask/strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_1/stack_1Ģ ?transformer_encoder/self_attention_mask/strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_1/stack_2Ü 7transformer_encoder/self_attention_mask/strided_slice_1 StridedSlice6transformer_encoder/self_attention_mask/Shape:output:0Ftransformer_encoder/self_attention_mask/strided_slice_1/stack:output:0Htransformer_encoder/self_attention_mask/strided_slice_1/stack_1:output:0Htransformer_encoder/self_attention_mask/strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask29 7transformer_encoder/self_attention_mask/strided_slice_1š /transformer_encoder/self_attention_mask/Shape_1Shapeinputs_1* T0* _output_shapes :21 /transformer_encoder/self_attention_mask/Shape_1Č =transformer_encoder/self_attention_mask/strided_slice_2/stackConst* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice_2/stackĢ ?transformer_encoder/self_attention_mask/strided_slice_2/stack_1Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_2/stack_1Ģ ?transformer_encoder/self_attention_mask/strided_slice_2/stack_2Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_2/stack_2Ž 7transformer_encoder/self_attention_mask/strided_slice_2 StridedSlice8transformer_encoder/self_attention_mask/Shape_1:output:0Ftransformer_encoder/self_attention_mask/strided_slice_2/stack:output:0Htransformer_encoder/self_attention_mask/strided_slice_2/stack_1:output:0Htransformer_encoder/self_attention_mask/strided_slice_2/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask29 7transformer_encoder/self_attention_mask/strided_slice_2Č =transformer_encoder/self_attention_mask/strided_slice_3/stackConst* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice_3/stackĢ ?transformer_encoder/self_attention_mask/strided_slice_3/stack_1Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_3/stack_1Ģ ?transformer_encoder/self_attention_mask/strided_slice_3/stack_2Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_3/stack_2Ž 7transformer_encoder/self_attention_mask/strided_slice_3 StridedSlice8transformer_encoder/self_attention_mask/Shape_1:output:0Ftransformer_encoder/self_attention_mask/strided_slice_3/stack:output:0Htransformer_encoder/self_attention_mask/strided_slice_3/stack_1:output:0Htransformer_encoder/self_attention_mask/strided_slice_3/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask29 7transformer_encoder/self_attention_mask/strided_slice_3“ 7transformer_encoder/self_attention_mask/Reshape/shape/1Const* _output_shapes :* dtype0* value B:29 7transformer_encoder/self_attention_mask/Reshape/shape/1č 5transformer_encoder/self_attention_mask/Reshape/shapePack>transformer_encoder/self_attention_mask/strided_slice:output:0@transformer_encoder/self_attention_mask/Reshape/shape/1:output:0@transformer_encoder/self_attention_mask/strided_slice_3:output:0* N* T0* _output_shapes :27 5transformer_encoder/self_attention_mask/Reshape/shapeö /transformer_encoder/self_attention_mask/ReshapeReshapeinputs_1>transformer_encoder/self_attention_mask/Reshape/shape:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’21 /transformer_encoder/self_attention_mask/Reshapeģ ,transformer_encoder/self_attention_mask/CastCast8transformer_encoder/self_attention_mask/Reshape:output:0* DstT0* SrcT0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2. ,transformer_encoder/self_attention_mask/CastŽ 0transformer_encoder/self_attention_mask/ones/mulMul>transformer_encoder/self_attention_mask/strided_slice:output:0@transformer_encoder/self_attention_mask/strided_slice_1:output:0* T0* _output_shapes :22 0transformer_encoder/self_attention_mask/ones/mul® 4transformer_encoder/self_attention_mask/ones/mul_1/yConst* _output_shapes :* dtype0* value B:26 4transformer_encoder/self_attention_mask/ones/mul_1/y… 2transformer_encoder/self_attention_mask/ones/mul_1Mul4transformer_encoder/self_attention_mask/ones/mul:z:0=transformer_encoder/self_attention_mask/ones/mul_1/y:output:0* T0* _output_shapes :24 2transformer_encoder/self_attention_mask/ones/mul_1­ 3transformer_encoder/self_attention_mask/ones/Less/yConst* _output_shapes :* dtype0* value B:č25 3transformer_encoder/self_attention_mask/ones/Less/y… 1transformer_encoder/self_attention_mask/ones/LessLess6transformer_encoder/self_attention_mask/ones/mul_1:z:0transformer_encoder/self_attention_mask/strided_slice:output:0@transformer_encoder/self_attention_mask/strided_slice_1:output:0>transformer_encoder/self_attention_mask/ones/packed/2:output:0* N* T0* _output_shapes :25 3transformer_encoder/self_attention_mask/ones/packed­ 2transformer_encoder/self_attention_mask/ones/ConstConst* _output_shapes :* dtype0* value B *€?24 2transformer_encoder/self_attention_mask/ones/Constž ,transformer_encoder/self_attention_mask/onesFillabde2L Jtransformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_0_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_0/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_0/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp— Htransformer_encoder/transformer/layer_0/self_attention/key/einsum/EinsumEinsum-transformer_encoder/dropout/Identity:output:0_transformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_0_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_0/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_0/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp Jtransformer_encoder/transformer/layer_0/self_attention/value/einsum/EinsumEinsum-transformer_encoder/dropout/Identity:output:0atransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_0_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_0/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_0/self_attention/value/add” Dtransformer_encoder/transformer/layer_0/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_0/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_0/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_0/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_0/self_attention/Mul:z:0Mtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_0/self_attention/masked_softmax/addÄ Mtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/SoftmaxSoftmaxMtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/SoftmaxĆ Gtransformer_encoder/transformer/layer_0/self_attention/dropout/IdentityIdentityWtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_0/self_attention/dropout/Identity Ftransformer_encoder/transformer/layer_0/self_attention/einsum_1/EinsumEinsumPtransformer_encoder/transformer/layer_0/self_attention/dropout/Identity:output:0Dtransformer_encoder/transformer/layer_0/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_0/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_0/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_0_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_0/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_0/self_attention/attention_output/add• :transformer_encoder/transformer/layer_0/dropout_1/IdentityIdentityOtransformer_encoder/transformer/layer_0/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_0/dropout_1/Identity— +transformer_encoder/transformer/layer_0/addAddV2-transformer_encoder/dropout/Identity:output:0Ctransformer_encoder/transformer/layer_0/dropout_1/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_0/addŽ `transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_0/add:z:0itransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_0/add:z:0_transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_0/add:z:0Stransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_0_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_0/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_0_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_0/intermediate/addAddV2Ktransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_0/intermediate/add¹ 8transformer_encoder/transformer/layer_0/activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2: 8transformer_encoder/transformer/layer_0/activation/Pow/yø 6transformer_encoder/transformer/layer_0/activation/PowPowabd2> abde2L Jtransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_1_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_1/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_1/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_1/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_1_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_1/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_1/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_1/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_1_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_1/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_1/self_attention/value/add” Dtransformer_encoder/transformer/layer_1/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_1/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_1/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_1/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_1/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/addŹ Otransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/SoftmaxÉ Itransformer_encoder/transformer/layer_1/self_attention/dropout_3/IdentityIdentityYtransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_1/self_attention/dropout_3/IdentityŸ Ftransformer_encoder/transformer/layer_1/self_attention/einsum_1/EinsumEinsumRtransformer_encoder/transformer/layer_1/self_attention/dropout_3/Identity:output:0Dtransformer_encoder/transformer/layer_1/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_1/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_1/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_1_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_1/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_1/self_attention/attention_output/add• :transformer_encoder/transformer/layer_1/dropout_4/IdentityIdentityOtransformer_encoder/transformer/layer_1/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_1/dropout_4/Identity· +transformer_encoder/transformer/layer_1/addAddV2Mtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Ctransformer_encoder/transformer/layer_1/dropout_4/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_1/addŽ `transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_1/add:z:0itransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_1/add:z:0_transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_1/add:z:0Stransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_1_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_1/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_1_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_1/intermediate/addAddV2Ktransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_1/intermediate/add½ :transformer_encoder/transformer/layer_1/activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_1/activation_1/Pow/y¾ 8transformer_encoder/transformer/layer_1/activation_1/PowPow transformer_encoder/transformer/layer_1/activation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_1/activation_1/TanhĮ transformer_encoder/transformer/layer_1/activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_1/activation_1/mul_2½ :transformer_encoder/transformer/layer_1/activation_1/mul_3Multransformer_encoder/transformer/layer_1/activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_1/activation_1/mul_3± Ktransformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_1_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_1/activation_1/mul_3:z:0Stransformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_2_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_2/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_2/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_2/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_2_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_2/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_2/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_2/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_2_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_2/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_2/self_attention/value/add” Dtransformer_encoder/transformer/layer_2/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_2/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_2/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_2/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_2/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/addŹ Otransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/SoftmaxÉ Itransformer_encoder/transformer/layer_2/self_attention/dropout_6/IdentityIdentityYtransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_2/self_attention/dropout_6/IdentityŸ Ftransformer_encoder/transformer/layer_2/self_attention/einsum_1/EinsumEinsumRtransformer_encoder/transformer/layer_2/self_attention/dropout_6/Identity:output:0Dtransformer_encoder/transformer/layer_2/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_2/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_2/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_2_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_2/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_2/self_attention/attention_output/add• :transformer_encoder/transformer/layer_2/dropout_7/IdentityIdentityOtransformer_encoder/transformer/layer_2/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_2/dropout_7/Identity· +transformer_encoder/transformer/layer_2/addAddV2Mtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Ctransformer_encoder/transformer/layer_2/dropout_7/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_2/addŽ `transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_2/add:z:0itransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_2/add:z:0_transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_2/add:z:0Stransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_2_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_2/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_2_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_2/intermediate/addAddV2Ktransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_2/intermediate/add½ :transformer_encoder/transformer/layer_2/activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_2/activation_2/Pow/y¾ 8transformer_encoder/transformer/layer_2/activation_2/PowPow transformer_encoder/transformer/layer_2/activation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_2/activation_2/TanhĮ transformer_encoder/transformer/layer_2/activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_2/activation_2/mul_2½ :transformer_encoder/transformer/layer_2/activation_2/mul_3Multransformer_encoder/transformer/layer_2/activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_2/activation_2/mul_3± Ktransformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_2_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_2/activation_2/mul_3:z:0Stransformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_3_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_3/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_3/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_3/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_3_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_3/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_3/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_3/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_3_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_3/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_3/self_attention/value/add” Dtransformer_encoder/transformer/layer_3/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_3/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_3/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_3/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_3/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/addŹ Otransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/SoftmaxÉ Itransformer_encoder/transformer/layer_3/self_attention/dropout_9/IdentityIdentityYtransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_3/self_attention/dropout_9/IdentityŸ Ftransformer_encoder/transformer/layer_3/self_attention/einsum_1/EinsumEinsumRtransformer_encoder/transformer/layer_3/self_attention/dropout_9/Identity:output:0Dtransformer_encoder/transformer/layer_3/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_3/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_3/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_3_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_3/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_3/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_3/dropout_10/IdentityIdentityOtransformer_encoder/transformer/layer_3/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_3/dropout_10/Identityø +transformer_encoder/transformer/layer_3/addAddV2Mtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_3/dropout_10/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_3/addŽ `transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_3/add:z:0itransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_3/add:z:0_transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_3/add:z:0Stransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_3_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_3/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_3_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_3/intermediate/addAddV2Ktransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_3/intermediate/add½ :transformer_encoder/transformer/layer_3/activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_3/activation_3/Pow/y¾ 8transformer_encoder/transformer/layer_3/activation_3/PowPow transformer_encoder/transformer/layer_3/activation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_3/activation_3/TanhĮ transformer_encoder/transformer/layer_3/activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_3/activation_3/mul_2½ :transformer_encoder/transformer/layer_3/activation_3/mul_3Multransformer_encoder/transformer/layer_3/activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_3/activation_3/mul_3± Ktransformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_3_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_3/activation_3/mul_3:z:0Stransformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_4_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_4/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_4/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_4/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_4_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_4/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_4/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_4/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_4_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_4/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_4/self_attention/value/add” Dtransformer_encoder/transformer/layer_4/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_4/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_4/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_4/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_4/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/addŹ Otransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/SoftmaxĖ Jtransformer_encoder/transformer/layer_4/self_attention/dropout_12/IdentityIdentityYtransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_4/self_attention/dropout_12/Identity  Ftransformer_encoder/transformer/layer_4/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_4/self_attention/dropout_12/Identity:output:0Dtransformer_encoder/transformer/layer_4/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_4/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_4/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_4_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_4/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_4/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_4/dropout_13/IdentityIdentityOtransformer_encoder/transformer/layer_4/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_4/dropout_13/Identityø +transformer_encoder/transformer/layer_4/addAddV2Mtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_4/dropout_13/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_4/addŽ `transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_4/add:z:0itransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_4/add:z:0_transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_4/add:z:0Stransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_4_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_4/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_4_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_4/intermediate/addAddV2Ktransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_4/intermediate/add½ :transformer_encoder/transformer/layer_4/activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_4/activation_4/Pow/y¾ 8transformer_encoder/transformer/layer_4/activation_4/PowPow transformer_encoder/transformer/layer_4/activation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_4/activation_4/TanhĮ transformer_encoder/transformer/layer_4/activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_4/activation_4/mul_2½ :transformer_encoder/transformer/layer_4/activation_4/mul_3Multransformer_encoder/transformer/layer_4/activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_4/activation_4/mul_3± Ktransformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_4_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_4/activation_4/mul_3:z:0Stransformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_5_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_5/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_5/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_5/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_5_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_5/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_5/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_5/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_5_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_5/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_5/self_attention/value/add” Dtransformer_encoder/transformer/layer_5/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_5/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_5/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_5/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_5/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/addŹ Otransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/SoftmaxĖ Jtransformer_encoder/transformer/layer_5/self_attention/dropout_15/IdentityIdentityYtransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_5/self_attention/dropout_15/Identity  Ftransformer_encoder/transformer/layer_5/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_5/self_attention/dropout_15/Identity:output:0Dtransformer_encoder/transformer/layer_5/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_5/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_5/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_5_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_5/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_5/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_5/dropout_16/IdentityIdentityOtransformer_encoder/transformer/layer_5/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_5/dropout_16/Identityø +transformer_encoder/transformer/layer_5/addAddV2Mtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_5/dropout_16/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_5/addŽ `transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_5/add:z:0itransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_5/add:z:0_transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_5/add:z:0Stransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_5_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_5/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_5_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_5/intermediate/addAddV2Ktransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_5/intermediate/add½ :transformer_encoder/transformer/layer_5/activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_5/activation_5/Pow/y¾ 8transformer_encoder/transformer/layer_5/activation_5/PowPow transformer_encoder/transformer/layer_5/activation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_5/activation_5/TanhĮ transformer_encoder/transformer/layer_5/activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_5/activation_5/mul_2½ :transformer_encoder/transformer/layer_5/activation_5/mul_3Multransformer_encoder/transformer/layer_5/activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_5/activation_5/mul_3± Ktransformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_5_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_5/activation_5/mul_3:z:0Stransformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_6_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_6/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_6/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_6/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_6_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_6/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_6/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_6/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_6_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_6/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_6/self_attention/value/add” Dtransformer_encoder/transformer/layer_6/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_6/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_6/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_6/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_6/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/addŹ Otransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/SoftmaxĖ Jtransformer_encoder/transformer/layer_6/self_attention/dropout_18/IdentityIdentityYtransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_6/self_attention/dropout_18/Identity  Ftransformer_encoder/transformer/layer_6/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_6/self_attention/dropout_18/Identity:output:0Dtransformer_encoder/transformer/layer_6/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_6/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_6/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_6_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_6/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_6/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_6/dropout_19/IdentityIdentityOtransformer_encoder/transformer/layer_6/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_6/dropout_19/Identityø +transformer_encoder/transformer/layer_6/addAddV2Mtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_6/dropout_19/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_6/addŽ `transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_6/add:z:0itransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_6/add:z:0_transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_6/add:z:0Stransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_6_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_6/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_6_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_6/intermediate/addAddV2Ktransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_6/intermediate/add½ :transformer_encoder/transformer/layer_6/activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_6/activation_6/Pow/y¾ 8transformer_encoder/transformer/layer_6/activation_6/PowPow transformer_encoder/transformer/layer_6/activation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_6/activation_6/TanhĮ transformer_encoder/transformer/layer_6/activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_6/activation_6/mul_2½ :transformer_encoder/transformer/layer_6/activation_6/mul_3Multransformer_encoder/transformer/layer_6/activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_6/activation_6/mul_3± Ktransformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_6_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_6/activation_6/mul_3:z:0Stransformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_7_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_7/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_7/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_7/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_7_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_7/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_7/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_7/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_7_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_7/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_7/self_attention/value/add” Dtransformer_encoder/transformer/layer_7/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_7/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_7/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_7/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_7/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/addŹ Otransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/SoftmaxĖ Jtransformer_encoder/transformer/layer_7/self_attention/dropout_21/IdentityIdentityYtransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_7/self_attention/dropout_21/Identity  Ftransformer_encoder/transformer/layer_7/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_7/self_attention/dropout_21/Identity:output:0Dtransformer_encoder/transformer/layer_7/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_7/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_7/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_7_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_7/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_7/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_7/dropout_22/IdentityIdentityOtransformer_encoder/transformer/layer_7/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_7/dropout_22/Identityø +transformer_encoder/transformer/layer_7/addAddV2Mtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_7/dropout_22/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_7/addŽ `transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_7/add:z:0itransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_7/add:z:0_transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_7/add:z:0Stransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_7_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_7/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_7_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_7/intermediate/addAddV2Ktransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_7/intermediate/add½ :transformer_encoder/transformer/layer_7/activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_7/activation_7/Pow/y¾ 8transformer_encoder/transformer/layer_7/activation_7/PowPow transformer_encoder/transformer/layer_7/activation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_7/activation_7/TanhĮ transformer_encoder/transformer/layer_7/activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_7/activation_7/mul_2½ :transformer_encoder/transformer/layer_7/activation_7/mul_3Multransformer_encoder/transformer/layer_7/activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_7/activation_7/mul_3± Ktransformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_7_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_7/activation_7/mul_3:z:0Stransformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_8_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_8/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_8/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_8/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_8_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_8/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_8/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_8/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_8_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_8/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_8/self_attention/value/add” Dtransformer_encoder/transformer/layer_8/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_8/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_8/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_8/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_8/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/addŹ Otransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/SoftmaxĖ Jtransformer_encoder/transformer/layer_8/self_attention/dropout_24/IdentityIdentityYtransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_8/self_attention/dropout_24/Identity  Ftransformer_encoder/transformer/layer_8/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_8/self_attention/dropout_24/Identity:output:0Dtransformer_encoder/transformer/layer_8/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_8/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_8/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_8_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_8/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_8/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_8/dropout_25/IdentityIdentityOtransformer_encoder/transformer/layer_8/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_8/dropout_25/Identityø +transformer_encoder/transformer/layer_8/addAddV2Mtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_8/dropout_25/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_8/addŽ `transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_8/add:z:0itransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_8/add:z:0_transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_8/add:z:0Stransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_8_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_8/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_8_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_8/intermediate/addAddV2Ktransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_8/intermediate/add½ :transformer_encoder/transformer/layer_8/activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_8/activation_8/Pow/y¾ 8transformer_encoder/transformer/layer_8/activation_8/PowPow transformer_encoder/transformer/layer_8/activation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_8/activation_8/TanhĮ transformer_encoder/transformer/layer_8/activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_8/activation_8/mul_2½ :transformer_encoder/transformer/layer_8/activation_8/mul_3Multransformer_encoder/transformer/layer_8/activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_8/activation_8/mul_3± Ktransformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_8_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_8/activation_8/mul_3:z:0Stransformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2L Jtransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_9_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_9/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_9/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_9/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_9_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_9/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_9/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_9/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_9_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_9/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_9/self_attention/value/add” Dtransformer_encoder/transformer/layer_9/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_9/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_9/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_9/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_9/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/addŹ Otransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/SoftmaxĖ Jtransformer_encoder/transformer/layer_9/self_attention/dropout_27/IdentityIdentityYtransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_9/self_attention/dropout_27/Identity  Ftransformer_encoder/transformer/layer_9/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_9/self_attention/dropout_27/Identity:output:0Dtransformer_encoder/transformer/layer_9/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_9/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_9/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_9_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_9/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_9/self_attention/attention_output/add— ;transformer_encoder/transformer/layer_9/dropout_28/IdentityIdentityOtransformer_encoder/transformer/layer_9/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer_encoder/transformer/layer_9/dropout_28/Identityø +transformer_encoder/transformer/layer_9/addAddV2Mtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_9/dropout_28/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_9/addŽ `transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_9/add:z:0itransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_9/add:z:0_transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_9/add:z:0Stransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_9_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_9/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_9_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_9/intermediate/addAddV2Ktransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_9/intermediate/add½ :transformer_encoder/transformer/layer_9/activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_9/activation_9/Pow/y¾ 8transformer_encoder/transformer/layer_9/activation_9/PowPow transformer_encoder/transformer/layer_9/activation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_9/activation_9/TanhĮ transformer_encoder/transformer/layer_9/activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_9/activation_9/mul_2½ :transformer_encoder/transformer/layer_9/activation_9/mul_3Multransformer_encoder/transformer/layer_9/activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_9/activation_9/mul_3± Ktransformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_9_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_9/activation_9/mul_3:z:0Stransformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> abde2M Ktransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_10_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_10/self_attention/query/addAddV2Ttransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_10/self_attention/query/addŪ Xtransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpatransformer_encoder_transformer_layer_10_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Z Xtransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOpŗ Itransformer_encoder/transformer/layer_10/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0`transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2K Itransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsumø Ntransformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOpReadVariableOpWtransformer_encoder_transformer_layer_10_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02P Ntransformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOpś ?transformer_encoder/transformer/layer_10/self_attention/key/addAddV2Rtransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum:output:0Vtransformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2A ?transformer_encoder/transformer/layer_10/self_attention/key/addį Ztransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02\ Ztransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpĄ Ktransformer_encoder/transformer/layer_10/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0btransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2M Ktransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_10_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_10/self_attention/value/addAddV2Ttransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_10/self_attention/value/add˜ Etransformer_encoder/transformer/layer_10/self_attention/einsum/EinsumEinsumCtransformer_encoder/transformer/layer_10/self_attention/key/add:z:0Etransformer_encoder/transformer/layer_10/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2G Etransformer_encoder/transformer/layer_10/self_attention/einsum/EinsumĆ =transformer_encoder/transformer/layer_10/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2? =transformer_encoder/transformer/layer_10/self_attention/Mul/yå ;transformer_encoder/transformer/layer_10/self_attention/MulMulNtransformer_encoder/transformer/layer_10/self_attention/einsum/Einsum:output:0Ftransformer_encoder/transformer/layer_10/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer_encoder/transformer/layer_10/self_attention/Mulž Xtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimš Ttransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims ExpandDims/transformer_encoder/self_attention_mask/mul:z:0atransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2V Ttransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDimsē Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2Q Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub/xŖ Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/subSubXtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub/x:output:0]transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/subē Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2Q Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul/yž Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mulMulQtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub:z:0Xtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul‡ Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/addAddV2?transformer_encoder/transformer/layer_10/self_attention/Mul:z:0Qtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/addŠ Qtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/SoftmaxSoftmaxQtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/SoftmaxĻ Ktransformer_encoder/transformer/layer_10/self_attention/dropout_30/IdentityIdentity[transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_10/self_attention/dropout_30/Identity¤ Gtransformer_encoder/transformer/layer_10/self_attention/einsum_1/EinsumEinsumTtransformer_encoder/transformer/layer_10/self_attention/dropout_30/Identity:output:0Etransformer_encoder/transformer/layer_10/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2I Gtransformer_encoder/transformer/layer_10/self_attention/einsum_1/Einsum‚ etransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpntransformer_encoder_transformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02g etransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpį Vtransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/EinsumEinsumPtransformer_encoder/transformer/layer_10/self_attention/einsum_1/Einsum:output:0mtransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2X Vtransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/EinsumÜ [transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOpReadVariableOpdtransformer_encoder_transformer_layer_10_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02] [transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOp« Ltransformer_encoder/transformer/layer_10/self_attention/attention_output/addAddV2_transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum:output:0ctransformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2N Ltransformer_encoder/transformer/layer_10/self_attention/attention_output/addš abd2E Ctransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum£ Htransformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOpReadVariableOpQtransformer_encoder_transformer_layer_10_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02J Htransformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOpß 9transformer_encoder/transformer/layer_10/intermediate/addAddV2Ltransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum:output:0Ptransformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_10/intermediate/addĮ transformer_encoder/transformer/layer_10/activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_10/activation_10/mulĄ :transformer_encoder/transformer/layer_10/activation_10/addAddV2=transformer_encoder/transformer/layer_10/intermediate/add:z:0>transformer_encoder/transformer/layer_10/activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_10/activation_10/addÅ >transformer_encoder/transformer/layer_10/activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2@ >transformer_encoder/transformer/layer_10/activation_10/mul_1/xĢ transformer_encoder/transformer/layer_10/activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2> transformer_encoder/transformer/layer_10/activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2@ >transformer_encoder/transformer/layer_10/activation_10/add_1/xĻ transformer_encoder/transformer/layer_10/activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2@ >transformer_encoder/transformer/layer_10/activation_10/mul_2/xĪ abd2? =transformer_encoder/transformer/layer_10/output/einsum/Einsum‘ Btransformer_encoder/transformer/layer_10/output/add/ReadVariableOpReadVariableOpKtransformer_encoder_transformer_layer_10_output_add_readvariableop_resource* _output_shapes :€* dtype02D Btransformer_encoder/transformer/layer_10/output/add/ReadVariableOpĒ 3transformer_encoder/transformer/layer_10/output/addAddV2Ftransformer_encoder/transformer/layer_10/output/einsum/Einsum:output:0Jtransformer_encoder/transformer/layer_10/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer_encoder/transformer/layer_10/output/add abde2M Ktransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_11_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_11/self_attention/query/addAddV2Ttransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_11/self_attention/query/addŪ Xtransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpatransformer_encoder_transformer_layer_11_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Z Xtransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOp» Itransformer_encoder/transformer/layer_11/self_attention/key/einsum/EinsumEinsumNtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0`transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2K Itransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsumø Ntransformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOpReadVariableOpWtransformer_encoder_transformer_layer_11_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02P Ntransformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOpś ?transformer_encoder/transformer/layer_11/self_attention/key/addAddV2Rtransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum:output:0Vtransformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2A ?transformer_encoder/transformer/layer_11/self_attention/key/addį Ztransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02\ Ztransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpĮ Ktransformer_encoder/transformer/layer_11/self_attention/value/einsum/EinsumEinsumNtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0btransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2M Ktransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_11_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_11/self_attention/value/addAddV2Ttransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_11/self_attention/value/add˜ Etransformer_encoder/transformer/layer_11/self_attention/einsum/EinsumEinsumCtransformer_encoder/transformer/layer_11/self_attention/key/add:z:0Etransformer_encoder/transformer/layer_11/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2G Etransformer_encoder/transformer/layer_11/self_attention/einsum/EinsumĆ =transformer_encoder/transformer/layer_11/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2? =transformer_encoder/transformer/layer_11/self_attention/Mul/yå ;transformer_encoder/transformer/layer_11/self_attention/MulMulNtransformer_encoder/transformer/layer_11/self_attention/einsum/Einsum:output:0Ftransformer_encoder/transformer/layer_11/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer_encoder/transformer/layer_11/self_attention/Mulž Xtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimš Ttransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims ExpandDims/transformer_encoder/self_attention_mask/mul:z:0atransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2V Ttransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDimsē Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2Q Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub/xŖ Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/subSubXtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub/x:output:0]transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/subē Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2Q Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul/yž Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mulMulQtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub:z:0Xtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul‡ Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/addAddV2?transformer_encoder/transformer/layer_11/self_attention/Mul:z:0Qtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/addŠ Qtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/SoftmaxSoftmaxQtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/SoftmaxĻ Ktransformer_encoder/transformer/layer_11/self_attention/dropout_33/IdentityIdentity[transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_11/self_attention/dropout_33/Identity¤ Gtransformer_encoder/transformer/layer_11/self_attention/einsum_1/EinsumEinsumTtransformer_encoder/transformer/layer_11/self_attention/dropout_33/Identity:output:0Etransformer_encoder/transformer/layer_11/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2I Gtransformer_encoder/transformer/layer_11/self_attention/einsum_1/Einsum‚ etransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpntransformer_encoder_transformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02g etransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpį Vtransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/EinsumEinsumPtransformer_encoder/transformer/layer_11/self_attention/einsum_1/Einsum:output:0mtransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2X Vtransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/EinsumÜ [transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOpReadVariableOpdtransformer_encoder_transformer_layer_11_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02] [transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOp« Ltransformer_encoder/transformer/layer_11/self_attention/attention_output/addAddV2_transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum:output:0ctransformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2N Ltransformer_encoder/transformer/layer_11/self_attention/attention_output/addš abd2E Ctransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum£ Htransformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOpReadVariableOpQtransformer_encoder_transformer_layer_11_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02J Htransformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOpß 9transformer_encoder/transformer/layer_11/intermediate/addAddV2Ltransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum:output:0Ptransformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_11/intermediate/addĮ transformer_encoder/transformer/layer_11/activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_11/activation_11/mulĄ :transformer_encoder/transformer/layer_11/activation_11/addAddV2=transformer_encoder/transformer/layer_11/intermediate/add:z:0>transformer_encoder/transformer/layer_11/activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_11/activation_11/addÅ >transformer_encoder/transformer/layer_11/activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2@ >transformer_encoder/transformer/layer_11/activation_11/mul_1/xĢ transformer_encoder/transformer/layer_11/activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2> transformer_encoder/transformer/layer_11/activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2@ >transformer_encoder/transformer/layer_11/activation_11/add_1/xĻ transformer_encoder/transformer/layer_11/activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2@ >transformer_encoder/transformer/layer_11/activation_11/mul_2/xĪ abd2? =transformer_encoder/transformer/layer_11/output/einsum/Einsum‘ Btransformer_encoder/transformer/layer_11/output/add/ReadVariableOpReadVariableOpKtransformer_encoder_transformer_layer_11_output_add_readvariableop_resource* _output_shapes :€* dtype02D Btransformer_encoder/transformer/layer_11/output/add/ReadVariableOpĒ 3transformer_encoder/transformer/layer_11/output/addAddV2Ftransformer_encoder/transformer/layer_11/output/einsum/Einsum:output:0Jtransformer_encoder/transformer/layer_11/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer_encoder/transformer/layer_11/output/add   _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : „ ļ 3__inference_transformer/layer_8_layer_call_fn_28302 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_108402 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ųŗ ” N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_11079 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_9/ExpandDims/dimõ *self_attention/masked_softmax_9/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_9/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_9/ExpandDims“ %self_attention/masked_softmax_9/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_9/sub/x‚ #self_attention/masked_softmax_9/subSub.self_attention/masked_softmax_9/sub/x:output:03self_attention/masked_softmax_9/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/sub“ %self_attention/masked_softmax_9/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_9/mul/yö #self_attention/masked_softmax_9/mulMul'self_attention/masked_softmax_9/sub:z:0.self_attention/masked_softmax_9/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/mulą #self_attention/masked_softmax_9/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/addŅ 'self_attention/masked_softmax_9/SoftmaxSoftmax'self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_9/Softmax— 'self_attention/dropout_27/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_27/dropout/Const† %self_attention/dropout_27/dropout/MulMul1self_attention/masked_softmax_9/Softmax:softmax:00self_attention/dropout_27/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_27/dropout/Mul³ 'self_attention/dropout_27/dropout/ShapeShape1self_attention/masked_softmax_9/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_27/dropout/Shapeœ >self_attention/dropout_27/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_27/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_27/dropout/random_uniform/RandomUniform© 0self_attention/dropout_27/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_27/dropout/GreaterEqual/yĄ .self_attention/dropout_27/dropout/GreaterEqual GreaterEqualGself_attention/dropout_27/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_27/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_27/dropout/GreaterEqualē &self_attention/dropout_27/dropout/CastCast2self_attention/dropout_27/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_27/dropout/Castü 'self_attention/dropout_27/dropout/Mul_1Mul)self_attention/dropout_27/dropout/Mul:z:0*self_attention/dropout_27/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_27/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_27/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_28/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_28/dropout/ConstĆ dropout_28/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_28/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/dropout/Mul‹ dropout_28/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_28/dropout/Shapeć /dropout_28/dropout/random_uniform/RandomUniform RandomUniform!dropout_28/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_28/dropout/random_uniform/RandomUniform‹ !dropout_28/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_28/dropout/GreaterEqual/yų dropout_28/dropout/GreaterEqual GreaterEqual8dropout_28/dropout/random_uniform/RandomUniform:output:0*dropout_28/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_28/dropout/GreaterEqual® dropout_28/dropout/CastCast#dropout_28/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/dropout/Cast“ dropout_28/dropout/Mul_1Muldropout_28/dropout/Mul:z:0dropout_28/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/dropout/Mul_1y addAddV2inputsdropout_28/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_9/Pow/yž activation_9/PowPowintermediate/add:z:0activation_9/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Powm activation_9/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_9/mul/xž activation_9/mulMulactivation_9/mul/x:output:0activation_9/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul™ activation_9/addAddV2intermediate/add:z:0activation_9/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/addq activation_9/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_9/mul_1/x¤ activation_9/mul_1Mulactivation_9/mul_1/x:output:0activation_9/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_1† activation_9/TanhTanhactivation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Tanhq activation_9/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_9/add_1/x§ activation_9/add_1AddV2activation_9/add_1/x:output:0activation_9/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/add_1q activation_9/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_9/mul_2/x¦ activation_9/mul_2Mulactivation_9/mul_2/x:output:0activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_2 activation_9/mul_3Mulintermediate/add:z:0activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_9/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_29/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_29/dropout/ConstŖ dropout_29/dropout/MulMuloutput/add:z:0!dropout_29/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/dropout/Mulr dropout_29/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_29/dropout/Shapeć /dropout_29/dropout/random_uniform/RandomUniform RandomUniform!dropout_29/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_29/dropout/random_uniform/RandomUniform‹ !dropout_29/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_29/dropout/GreaterEqual/yų dropout_29/dropout/GreaterEqual GreaterEqual8dropout_29/dropout/random_uniform/RandomUniform:output:0*dropout_29/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_29/dropout/GreaterEqual® dropout_29/dropout/CastCast#dropout_29/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/dropout/Cast“ dropout_29/dropout/Mul_1Muldropout_29/dropout/Mul:z:0dropout_29/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/dropout/Mul_1¤ add_1AddV2dropout_29/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ↠ļ7 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12906 inputs inputs_1 inputs_2 word_embeddings_12485 position_embedding_12488 type_embeddings_12491 embeddings_layer_norm_12495 embeddings_layer_norm_12497 transformer_layer_0_12502 transformer_layer_0_12504 transformer_layer_0_12506 transformer_layer_0_12508 transformer_layer_0_12510 transformer_layer_0_12512 transformer_layer_0_12514 transformer_layer_0_12516 transformer_layer_0_12518 transformer_layer_0_12520 transformer_layer_0_12522 transformer_layer_0_12524 transformer_layer_0_12526 transformer_layer_0_12528 transformer_layer_0_12530 transformer_layer_0_12532 transformer_layer_1_12535 transformer_layer_1_12537 transformer_layer_1_12539 transformer_layer_1_12541 transformer_layer_1_12543 transformer_layer_1_12545 transformer_layer_1_12547 transformer_layer_1_12549 transformer_layer_1_12551 transformer_layer_1_12553 transformer_layer_1_12555 transformer_layer_1_12557 transformer_layer_1_12559 transformer_layer_1_12561 transformer_layer_1_12563 transformer_layer_1_12565 transformer_layer_2_12568 transformer_layer_2_12570 transformer_layer_2_12572 transformer_layer_2_12574 transformer_layer_2_12576 transformer_layer_2_12578 transformer_layer_2_12580 transformer_layer_2_12582 transformer_layer_2_12584 transformer_layer_2_12586 transformer_layer_2_12588 transformer_layer_2_12590 transformer_layer_2_12592 transformer_layer_2_12594 transformer_layer_2_12596 transformer_layer_2_12598 transformer_layer_3_12601 transformer_layer_3_12603 transformer_layer_3_12605 transformer_layer_3_12607 transformer_layer_3_12609 transformer_layer_3_12611 transformer_layer_3_12613 transformer_layer_3_12615 transformer_layer_3_12617 transformer_layer_3_12619 transformer_layer_3_12621 transformer_layer_3_12623 transformer_layer_3_12625 transformer_layer_3_12627 transformer_layer_3_12629 transformer_layer_3_12631 transformer_layer_4_12634 transformer_layer_4_12636 transformer_layer_4_12638 transformer_layer_4_12640 transformer_layer_4_12642 transformer_layer_4_12644 transformer_layer_4_12646 transformer_layer_4_12648 transformer_layer_4_12650 transformer_layer_4_12652 transformer_layer_4_12654 transformer_layer_4_12656 transformer_layer_4_12658 transformer_layer_4_12660 transformer_layer_4_12662 transformer_layer_4_12664 transformer_layer_5_12667 transformer_layer_5_12669 transformer_layer_5_12671 transformer_layer_5_12673 transformer_layer_5_12675 transformer_layer_5_12677 transformer_layer_5_12679 transformer_layer_5_12681 transformer_layer_5_12683 transformer_layer_5_12685 transformer_layer_5_12687 transformer_layer_5_12689 transformer_layer_5_12691 transformer_layer_5_12693 transformer_layer_5_12695 transformer_layer_5_12697 transformer_layer_6_12700 transformer_layer_6_12702 transformer_layer_6_12704 transformer_layer_6_12706 transformer_layer_6_12708 transformer_layer_6_12710 transformer_layer_6_12712 transformer_layer_6_12714 transformer_layer_6_12716 transformer_layer_6_12718 transformer_layer_6_12720 transformer_layer_6_12722 transformer_layer_6_12724 transformer_layer_6_12726 transformer_layer_6_12728 transformer_layer_6_12730 transformer_layer_7_12733 transformer_layer_7_12735 transformer_layer_7_12737 transformer_layer_7_12739 transformer_layer_7_12741 transformer_layer_7_12743 transformer_layer_7_12745 transformer_layer_7_12747 transformer_layer_7_12749 transformer_layer_7_12751 transformer_layer_7_12753 transformer_layer_7_12755 transformer_layer_7_12757 transformer_layer_7_12759 transformer_layer_7_12761 transformer_layer_7_12763 transformer_layer_8_12766 transformer_layer_8_12768 transformer_layer_8_12770 transformer_layer_8_12772 transformer_layer_8_12774 transformer_layer_8_12776 transformer_layer_8_12778 transformer_layer_8_12780 transformer_layer_8_12782 transformer_layer_8_12784 transformer_layer_8_12786 transformer_layer_8_12788 transformer_layer_8_12790 transformer_layer_8_12792 transformer_layer_8_12794 transformer_layer_8_12796 transformer_layer_9_12799 transformer_layer_9_12801 transformer_layer_9_12803 transformer_layer_9_12805 transformer_layer_9_12807 transformer_layer_9_12809 transformer_layer_9_12811 transformer_layer_9_12813 transformer_layer_9_12815 transformer_layer_9_12817 transformer_layer_9_12819 transformer_layer_9_12821 transformer_layer_9_12823 transformer_layer_9_12825 transformer_layer_9_12827 transformer_layer_9_12829 transformer_layer_10_12832 transformer_layer_10_12834 transformer_layer_10_12836 transformer_layer_10_12838 transformer_layer_10_12840 transformer_layer_10_12842 transformer_layer_10_12844 transformer_layer_10_12846 transformer_layer_10_12848 transformer_layer_10_12850 transformer_layer_10_12852 transformer_layer_10_12854 transformer_layer_10_12856 transformer_layer_10_12858 transformer_layer_10_12860 transformer_layer_10_12862 transformer_layer_11_12865 transformer_layer_11_12867 transformer_layer_11_12869 transformer_layer_11_12871 transformer_layer_11_12873 transformer_layer_11_12875 transformer_layer_11_12877 transformer_layer_11_12879 transformer_layer_11_12881 transformer_layer_11_12883 transformer_layer_11_12885 transformer_layer_11_12887 transformer_layer_11_12889 transformer_layer_11_12891 transformer_layer_11_12893 transformer_layer_11_12895 pooler_transform_12899 pooler_transform_12901 identity identity_1ˆ¢dropout/StatefulPartitionedCall¢-embeddings/layer_norm/StatefulPartitionedCall¢(pooler_transform/StatefulPartitionedCall¢*position_embedding/StatefulPartitionedCall¢+transformer/layer_0/StatefulPartitionedCall¢+transformer/layer_1/StatefulPartitionedCall¢,transformer/layer_10/StatefulPartitionedCall¢,transformer/layer_11/StatefulPartitionedCall¢+transformer/layer_2/StatefulPartitionedCall¢+transformer/layer_3/StatefulPartitionedCall¢+transformer/layer_4/StatefulPartitionedCall¢+transformer/layer_5/StatefulPartitionedCall¢+transformer/layer_6/StatefulPartitionedCall¢+transformer/layer_7/StatefulPartitionedCall¢+transformer/layer_8/StatefulPartitionedCall¢+transformer/layer_9/StatefulPartitionedCall¢'type_embeddings/StatefulPartitionedCall¢'word_embeddings/StatefulPartitionedCall„ 'word_embeddings/StatefulPartitionedCallStatefulPartitionedCallinputsword_embeddings_12485* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_word_embeddings_layer_call_and_return_conditional_losses_76282) 'word_embeddings/StatefulPartitionedCallŪ *position_embedding/StatefulPartitionedCallStatefulPartitionedCall0word_embeddings/StatefulPartitionedCall:output:0position_embedding_12488* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*U fPRN L__inference_position_embedding_layer_call_and_return_conditional_losses_76632, *position_embedding/StatefulPartitionedCall§ 'type_embeddings/StatefulPartitionedCallStatefulPartitionedCallinputs_2type_embeddings_12491* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_type_embeddings_layer_call_and_return_conditional_losses_76932) 'type_embeddings/StatefulPartitionedCallį add/PartitionedCallPartitionedCall0word_embeddings/StatefulPartitionedCall:output:03position_embedding/StatefulPartitionedCall:output:00type_embeddings/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*F fAR? =__inference_add_layer_call_and_return_conditional_losses_77132 add/PartitionedCallņ -embeddings/layer_norm/StatefulPartitionedCallStatefulPartitionedCalladd/PartitionedCall:output:0embeddings_layer_norm_12495embeddings_layer_norm_12497* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*X fSRQ O__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_77452/ -embeddings/layer_norm/StatefulPartitionedCall¢ dropout/StatefulPartitionedCallStatefulPartitionedCall6embeddings/layer_norm/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_dropout_layer_call_and_return_conditional_losses_77732! dropout/StatefulPartitionedCall³ #self_attention_mask/PartitionedCallPartitionedCall(dropout/StatefulPartitionedCall:output:0inputs_1* Tin 2* Tout 2* _collective_manager_ids *= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*V fQRO M__inference_self_attention_mask_layer_call_and_return_conditional_losses_78282% #self_attention_mask/PartitionedCall¹ +transformer/layer_0/StatefulPartitionedCallStatefulPartitionedCall(dropout/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_0_12502transformer_layer_0_12504transformer_layer_0_12506transformer_layer_0_12508transformer_layer_0_12510transformer_layer_0_12512transformer_layer_0_12514transformer_layer_0_12516transformer_layer_0_12518transformer_layer_0_12520transformer_layer_0_12522transformer_layer_0_12524transformer_layer_0_12526transformer_layer_0_12528transformer_layer_0_12530transformer_layer_0_12532* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_79652- +transformer/layer_0/StatefulPartitionedCallÅ +transformer/layer_1/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_0/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_1_12535transformer_layer_1_12537transformer_layer_1_12539transformer_layer_1_12541transformer_layer_1_12543transformer_layer_1_12545transformer_layer_1_12547transformer_layer_1_12549transformer_layer_1_12551transformer_layer_1_12553transformer_layer_1_12555transformer_layer_1_12557transformer_layer_1_12559transformer_layer_1_12561transformer_layer_1_12563transformer_layer_1_12565* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_83112- +transformer/layer_1/StatefulPartitionedCallÅ +transformer/layer_2/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_1/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_2_12568transformer_layer_2_12570transformer_layer_2_12572transformer_layer_2_12574transformer_layer_2_12576transformer_layer_2_12578transformer_layer_2_12580transformer_layer_2_12582transformer_layer_2_12584transformer_layer_2_12586transformer_layer_2_12588transformer_layer_2_12590transformer_layer_2_12592transformer_layer_2_12594transformer_layer_2_12596transformer_layer_2_12598* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_86572- +transformer/layer_2/StatefulPartitionedCallÅ +transformer/layer_3/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_2/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_3_12601transformer_layer_3_12603transformer_layer_3_12605transformer_layer_3_12607transformer_layer_3_12609transformer_layer_3_12611transformer_layer_3_12613transformer_layer_3_12615transformer_layer_3_12617transformer_layer_3_12619transformer_layer_3_12621transformer_layer_3_12623transformer_layer_3_12625transformer_layer_3_12627transformer_layer_3_12629transformer_layer_3_12631* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_90032- +transformer/layer_3/StatefulPartitionedCallÅ +transformer/layer_4/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_3/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_4_12634transformer_layer_4_12636transformer_layer_4_12638transformer_layer_4_12640transformer_layer_4_12642transformer_layer_4_12644transformer_layer_4_12646transformer_layer_4_12648transformer_layer_4_12650transformer_layer_4_12652transformer_layer_4_12654transformer_layer_4_12656transformer_layer_4_12658transformer_layer_4_12660transformer_layer_4_12662transformer_layer_4_12664* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_93492- +transformer/layer_4/StatefulPartitionedCallÅ +transformer/layer_5/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_4/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_5_12667transformer_layer_5_12669transformer_layer_5_12671transformer_layer_5_12673transformer_layer_5_12675transformer_layer_5_12677transformer_layer_5_12679transformer_layer_5_12681transformer_layer_5_12683transformer_layer_5_12685transformer_layer_5_12687transformer_layer_5_12689transformer_layer_5_12691transformer_layer_5_12693transformer_layer_5_12695transformer_layer_5_12697* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_96952- +transformer/layer_5/StatefulPartitionedCallĘ +transformer/layer_6/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_5/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_6_12700transformer_layer_6_12702transformer_layer_6_12704transformer_layer_6_12706transformer_layer_6_12708transformer_layer_6_12710transformer_layer_6_12712transformer_layer_6_12714transformer_layer_6_12716transformer_layer_6_12718transformer_layer_6_12720transformer_layer_6_12722transformer_layer_6_12724transformer_layer_6_12726transformer_layer_6_12728transformer_layer_6_12730* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_100412- +transformer/layer_6/StatefulPartitionedCallĘ +transformer/layer_7/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_6/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_7_12733transformer_layer_7_12735transformer_layer_7_12737transformer_layer_7_12739transformer_layer_7_12741transformer_layer_7_12743transformer_layer_7_12745transformer_layer_7_12747transformer_layer_7_12749transformer_layer_7_12751transformer_layer_7_12753transformer_layer_7_12755transformer_layer_7_12757transformer_layer_7_12759transformer_layer_7_12761transformer_layer_7_12763* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_103872- +transformer/layer_7/StatefulPartitionedCallĘ +transformer/layer_8/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_7/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_8_12766transformer_layer_8_12768transformer_layer_8_12770transformer_layer_8_12772transformer_layer_8_12774transformer_layer_8_12776transformer_layer_8_12778transformer_layer_8_12780transformer_layer_8_12782transformer_layer_8_12784transformer_layer_8_12786transformer_layer_8_12788transformer_layer_8_12790transformer_layer_8_12792transformer_layer_8_12794transformer_layer_8_12796* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_107332- +transformer/layer_8/StatefulPartitionedCallĘ +transformer/layer_9/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_8/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_9_12799transformer_layer_9_12801transformer_layer_9_12803transformer_layer_9_12805transformer_layer_9_12807transformer_layer_9_12809transformer_layer_9_12811transformer_layer_9_12813transformer_layer_9_12815transformer_layer_9_12817transformer_layer_9_12819transformer_layer_9_12821transformer_layer_9_12823transformer_layer_9_12825transformer_layer_9_12827transformer_layer_9_12829* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_110792- +transformer/layer_9/StatefulPartitionedCallŁ ,transformer/layer_10/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_9/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_10_12832transformer_layer_10_12834transformer_layer_10_12836transformer_layer_10_12838transformer_layer_10_12840transformer_layer_10_12842transformer_layer_10_12844transformer_layer_10_12846transformer_layer_10_12848transformer_layer_10_12850transformer_layer_10_12852transformer_layer_10_12854transformer_layer_10_12856transformer_layer_10_12858transformer_layer_10_12860transformer_layer_10_12862* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_114252. ,transformer/layer_10/StatefulPartitionedCallŚ ,transformer/layer_11/StatefulPartitionedCallStatefulPartitionedCall5transformer/layer_10/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_11_12865transformer_layer_11_12867transformer_layer_11_12869transformer_layer_11_12871transformer_layer_11_12873transformer_layer_11_12875transformer_layer_11_12877transformer_layer_11_12879transformer_layer_11_12881transformer_layer_11_12883transformer_layer_11_12885transformer_layer_11_12887transformer_layer_11_12889transformer_layer_11_12891transformer_layer_11_12893transformer_layer_11_12895* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_117712. ,transformer/layer_11/StatefulPartitionedCallś lambda/PartitionedCallPartitionedCall5transformer/layer_11/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_lambda_layer_call_and_return_conditional_losses_119982 lambda/PartitionedCallŠ (pooler_transform/StatefulPartitionedCallStatefulPartitionedCalllambda/PartitionedCall:output:0pooler_transform_12899pooler_transform_12901* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*T fORM K__inference_pooler_transform_layer_call_and_return_conditional_losses_120312* (pooler_transform/StatefulPartitionedCallæ IdentityIdentity5transformer/layer_11/StatefulPartitionedCall:output:0 ^dropout/StatefulPartitionedCall.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity² Identity_1Identity1pooler_transform/StatefulPartitionedCall:output:0 ^dropout/StatefulPartitionedCall.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2B dropout/StatefulPartitionedCalldropout/StatefulPartitionedCall2^ -embeddings/layer_norm/StatefulPartitionedCall-embeddings/layer_norm/StatefulPartitionedCall2T (pooler_transform/StatefulPartitionedCall(pooler_transform/StatefulPartitionedCall2X *position_embedding/StatefulPartitionedCall*position_embedding/StatefulPartitionedCall2Z +transformer/layer_0/StatefulPartitionedCall+transformer/layer_0/StatefulPartitionedCall2Z +transformer/layer_1/StatefulPartitionedCall+transformer/layer_1/StatefulPartitionedCall2\ ,transformer/layer_10/StatefulPartitionedCall,transformer/layer_10/StatefulPartitionedCall2\ ,transformer/layer_11/StatefulPartitionedCall,transformer/layer_11/StatefulPartitionedCall2Z +transformer/layer_2/StatefulPartitionedCall+transformer/layer_2/StatefulPartitionedCall2Z +transformer/layer_3/StatefulPartitionedCall+transformer/layer_3/StatefulPartitionedCall2Z +transformer/layer_4/StatefulPartitionedCall+transformer/layer_4/StatefulPartitionedCall2Z +transformer/layer_5/StatefulPartitionedCall+transformer/layer_5/StatefulPartitionedCall2Z +transformer/layer_6/StatefulPartitionedCall+transformer/layer_6/StatefulPartitionedCall2Z +transformer/layer_7/StatefulPartitionedCall+transformer/layer_7/StatefulPartitionedCall2Z +transformer/layer_8/StatefulPartitionedCall+transformer/layer_8/StatefulPartitionedCall2Z +transformer/layer_9/StatefulPartitionedCall+transformer/layer_9/StatefulPartitionedCall2R 'type_embeddings/StatefulPartitionedCall'type_embeddings/StatefulPartitionedCall2R 'word_embeddings/StatefulPartitionedCall'word_embeddings/StatefulPartitionedCall:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ‹q ę ,__inference_functional_1_layer_call_fn_17400 input_word_ids input_mask input_type_ids unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall® StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idsunknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*P fKRI G__inference_functional_1_layer_call_and_return_conditional_losses_169972 StatefulPartitionedCall IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity  Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ųŗ ” N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_10733 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_8/ExpandDims/dimõ *self_attention/masked_softmax_8/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_8/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_8/ExpandDims“ %self_attention/masked_softmax_8/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_8/sub/x‚ #self_attention/masked_softmax_8/subSub.self_attention/masked_softmax_8/sub/x:output:03self_attention/masked_softmax_8/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/sub“ %self_attention/masked_softmax_8/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_8/mul/yö #self_attention/masked_softmax_8/mulMul'self_attention/masked_softmax_8/sub:z:0.self_attention/masked_softmax_8/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/mulą #self_attention/masked_softmax_8/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/addŅ 'self_attention/masked_softmax_8/SoftmaxSoftmax'self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_8/Softmax— 'self_attention/dropout_24/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_24/dropout/Const† %self_attention/dropout_24/dropout/MulMul1self_attention/masked_softmax_8/Softmax:softmax:00self_attention/dropout_24/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_24/dropout/Mul³ 'self_attention/dropout_24/dropout/ShapeShape1self_attention/masked_softmax_8/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_24/dropout/Shapeœ >self_attention/dropout_24/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_24/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_24/dropout/random_uniform/RandomUniform© 0self_attention/dropout_24/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_24/dropout/GreaterEqual/yĄ .self_attention/dropout_24/dropout/GreaterEqual GreaterEqualGself_attention/dropout_24/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_24/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_24/dropout/GreaterEqualē &self_attention/dropout_24/dropout/CastCast2self_attention/dropout_24/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_24/dropout/Castü 'self_attention/dropout_24/dropout/Mul_1Mul)self_attention/dropout_24/dropout/Mul:z:0*self_attention/dropout_24/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_24/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_24/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_25/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_25/dropout/ConstĆ dropout_25/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_25/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/dropout/Mul‹ dropout_25/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_25/dropout/Shapeć /dropout_25/dropout/random_uniform/RandomUniform RandomUniform!dropout_25/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_25/dropout/random_uniform/RandomUniform‹ !dropout_25/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_25/dropout/GreaterEqual/yų dropout_25/dropout/GreaterEqual GreaterEqual8dropout_25/dropout/random_uniform/RandomUniform:output:0*dropout_25/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_25/dropout/GreaterEqual® dropout_25/dropout/CastCast#dropout_25/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/dropout/Cast“ dropout_25/dropout/Mul_1Muldropout_25/dropout/Mul:z:0dropout_25/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/dropout/Mul_1y addAddV2inputsdropout_25/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_8/Pow/yž activation_8/PowPowintermediate/add:z:0activation_8/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Powm activation_8/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_8/mul/xž activation_8/mulMulactivation_8/mul/x:output:0activation_8/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul™ activation_8/addAddV2intermediate/add:z:0activation_8/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/addq activation_8/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_8/mul_1/x¤ activation_8/mul_1Mulactivation_8/mul_1/x:output:0activation_8/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_1† activation_8/TanhTanhactivation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Tanhq activation_8/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_8/add_1/x§ activation_8/add_1AddV2activation_8/add_1/x:output:0activation_8/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/add_1q activation_8/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_8/mul_2/x¦ activation_8/mul_2Mulactivation_8/mul_2/x:output:0activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_2 activation_8/mul_3Mulintermediate/add:z:0activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_8/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_26/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_26/dropout/ConstŖ dropout_26/dropout/MulMuloutput/add:z:0!dropout_26/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/dropout/Mulr dropout_26/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_26/dropout/Shapeć /dropout_26/dropout/random_uniform/RandomUniform RandomUniform!dropout_26/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_26/dropout/random_uniform/RandomUniform‹ !dropout_26/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_26/dropout/GreaterEqual/yų dropout_26/dropout/GreaterEqual GreaterEqual8dropout_26/dropout/random_uniform/RandomUniform:output:0*dropout_26/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_26/dropout/GreaterEqual® dropout_26/dropout/CastCast#dropout_26/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/dropout/Cast“ dropout_26/dropout/Mul_1Muldropout_26/dropout/Mul:z:0dropout_26/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/dropout/Mul_1¤ add_1AddV2dropout_26/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ą  J__inference_type_embeddings_layer_call_and_return_conditional_losses_25380 inputs" matmul_readvariableop_resource identityˆq Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’2 Reshape/shapek ReshapeReshapeinputsReshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2 Reshapei one_hot/on_valueConst* _output_shapes :* dtype0* value B *€?2 one_hot/on_valuek one_hot/off_valueConst* _output_shapes :* dtype0* value B *2 one_hot/off_value` one_hot/depthConst* _output_shapes :* dtype0* value B:2 one_hot/depth¹ one_hotOneHotReshape:output:0one_hot/depth:output:0one_hot/on_value:output:0one_hot/off_value:output:0* T0* TI0*' _output_shapes : ’’’’’’’’’2 one_hotŽ MatMul/ReadVariableOpReadVariableOpmatmul_readvariableop_resource* _output_shapes : €* dtype02 MatMul/ReadVariableOp~ MatMulMatMulone_hot:output:0MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2 MatMulD ShapeShapeinputs* T0* _output_shapes :2 Shapem concat/values_1Const* _output_shapes :* dtype0* valueB :€2 concat/values_1\ concat/axisConst* _output_shapes :* dtype0* value B:2 concat/axisŠ concatConcatV2Shape:output:0concat/values_1:output:0concat/axis:output:0* N* T0* _output_shapes :2 concat„ Reshape_1ReshapeMatMul:product:0concat:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Reshape_1t IdentityIdentityReshape_1:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*3 _input_shapes" : ’’’’’’’’’ ’’’’’’’’’::XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes :   x 2__inference_position_embedding_layer_call_fn_25362 inputs unknown identityˆ¢StatefulPartitionedCallś StatefulPartitionedCallStatefulPartitionedCallinputsunknown* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*U fPRN L__inference_position_embedding_layer_call_and_return_conditional_losses_76632 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*8 _input_shapes' %: ’’’’’’’’’ ’’’’’’’’’€:22 StatefulPartitionedCallStatefulPartitionedCall:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes : ļp ß 3__inference_transformer_encoder_layer_call_fn_25311 inputs_0 inputs_1 inputs_2 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall§ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1inputs_2unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_137392 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity“ Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/2:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : š ” N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_10840 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_8/ExpandDims/dimõ *self_attention/masked_softmax_8/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_8/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_8/ExpandDims“ %self_attention/masked_softmax_8/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_8/sub/x‚ #self_attention/masked_softmax_8/subSub.self_attention/masked_softmax_8/sub/x:output:03self_attention/masked_softmax_8/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/sub“ %self_attention/masked_softmax_8/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_8/mul/yö #self_attention/masked_softmax_8/mulMul'self_attention/masked_softmax_8/sub:z:0.self_attention/masked_softmax_8/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/mulą #self_attention/masked_softmax_8/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/addŅ 'self_attention/masked_softmax_8/SoftmaxSoftmax'self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_8/SoftmaxÓ "self_attention/dropout_24/IdentityIdentity1self_attention/masked_softmax_8/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_24/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_24/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_25/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/Identityy addAddV2inputsdropout_25/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_8/Pow/yž activation_8/PowPowintermediate/add:z:0activation_8/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Powm activation_8/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_8/mul/xž activation_8/mulMulactivation_8/mul/x:output:0activation_8/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul™ activation_8/addAddV2intermediate/add:z:0activation_8/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/addq activation_8/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_8/mul_1/x¤ activation_8/mul_1Mulactivation_8/mul_1/x:output:0activation_8/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_1† activation_8/TanhTanhactivation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Tanhq activation_8/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_8/add_1/x§ activation_8/add_1AddV2activation_8/add_1/x:output:0activation_8/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/add_1q activation_8/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_8/mul_2/x¦ activation_8/mul_2Mulactivation_8/mul_2/x:output:0activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_2 activation_8/mul_3Mulintermediate/add:z:0activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_8/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_26/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/Identity¤ add_1AddV2dropout_26/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ńö ū¬ __inference__wrapped_model_7608 input_word_ids input_mask input_type_idsD @functional_1_transformer_encoder_word_embeddings_gather_resourceO Kfunctional_1_transformer_encoder_position_embedding_readvariableop_resourceS Ofunctional_1_transformer_encoder_type_embeddings_matmul_readvariableop_resource` \functional_1_transformer_encoder_embeddings_layer_norm_batchnorm_mul_readvariableop_resource\ Xfunctional_1_transformer_encoder_embeddings_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_0_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_0_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_0_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_0_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_0_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_0_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_0_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_0_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_0_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_1_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_1_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_1_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_1_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_1_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_1_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_1_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_1_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_2_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_2_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_2_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_2_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_2_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_2_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_2_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_2_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_3_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_3_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_3_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_3_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_3_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_3_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_3_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_3_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_4_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_4_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_4_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_4_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_4_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_4_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_4_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_4_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_5_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_5_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_5_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_5_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_5_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_5_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_5_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_5_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_6_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_6_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_6_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_6_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_6_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_6_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_6_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_6_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_7_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_7_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_7_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_7_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_7_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_7_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_7_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_7_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_8_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_8_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_8_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_8_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_8_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_8_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_8_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_8_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_9_self_attention_query_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resourceg cfunctional_1_transformer_encoder_transformer_layer_9_self_attention_key_add_readvariableop_resources ofunctional_1_transformer_encoder_transformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resourcei efunctional_1_transformer_encoder_transformer_layer_9_self_attention_value_add_readvariableop_resource~ zfunctional_1_transformer_encoder_transformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_9_self_attention_attention_output_add_readvariableop_resourcex tfunctional_1_transformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resourcek gfunctional_1_transformer_encoder_transformer_layer_9_intermediate_einsum_einsum_readvariableop_resourcea ]functional_1_transformer_encoder_transformer_layer_9_intermediate_add_readvariableop_resourcee afunctional_1_transformer_encoder_transformer_layer_9_output_einsum_einsum_readvariableop_resource[ Wfunctional_1_transformer_encoder_transformer_layer_9_output_add_readvariableop_resourcep lfunctional_1_transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resourcej ffunctional_1_transformer_encoder_transformer_layer_10_self_attention_query_add_readvariableop_resourcer nfunctional_1_transformer_encoder_transformer_layer_10_self_attention_key_einsum_einsum_readvariableop_resourceh dfunctional_1_transformer_encoder_transformer_layer_10_self_attention_key_add_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resourcej ffunctional_1_transformer_encoder_transformer_layer_10_self_attention_value_add_readvariableop_resource {functional_1_transformer_encoder_transformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resourceu qfunctional_1_transformer_encoder_transformer_layer_10_self_attention_attention_output_add_readvariableop_resourcey ufunctional_1_transformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceu qfunctional_1_transformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_10_intermediate_einsum_einsum_readvariableop_resourceb ^functional_1_transformer_encoder_transformer_layer_10_intermediate_add_readvariableop_resourcef bfunctional_1_transformer_encoder_transformer_layer_10_output_einsum_einsum_readvariableop_resource\ Xfunctional_1_transformer_encoder_transformer_layer_10_output_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_mul_readvariableop_resourcem ifunctional_1_transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_11_self_attention_query_einsum_einsum_readvariableop_resourcej ffunctional_1_transformer_encoder_transformer_layer_11_self_attention_query_add_readvariableop_resourcer nfunctional_1_transformer_encoder_transformer_layer_11_self_attention_key_einsum_einsum_readvariableop_resourceh dfunctional_1_transformer_encoder_transformer_layer_11_self_attention_key_add_readvariableop_resourcet pfunctional_1_transformer_encoder_transformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resourcej ffunctional_1_transformer_encoder_transformer_layer_11_self_attention_value_add_readvariableop_resource {functional_1_transformer_encoder_transformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resourceu qfunctional_1_transformer_encoder_transformer_layer_11_self_attention_attention_output_add_readvariableop_resourcey ufunctional_1_transformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceu qfunctional_1_transformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resourcel hfunctional_1_transformer_encoder_transformer_layer_11_intermediate_einsum_einsum_readvariableop_resourceb ^functional_1_transformer_encoder_transformer_layer_11_intermediate_add_readvariableop_resourcef bfunctional_1_transformer_encoder_transformer_layer_11_output_einsum_einsum_readvariableop_resource\ Xfunctional_1_transformer_encoder_transformer_layer_11_output_add_readvariableop_resourceq mfunctional_1_transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_mul_readvariableop_resourcem ifunctional_1_transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_readvariableop_resourceT Pfunctional_1_transformer_encoder_pooler_transform_matmul_readvariableop_resourceU Qfunctional_1_transformer_encoder_pooler_transform_biasadd_readvariableop_resource identity identity_1ˆÓ >functional_1/transformer_encoder/word_embeddings/Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’2@ >functional_1/transformer_encoder/word_embeddings/Reshape/shape† 8functional_1/transformer_encoder/word_embeddings/ReshapeReshapeinput_word_idsGfunctional_1/transformer_encoder/word_embeddings/Reshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2: 8functional_1/transformer_encoder/word_embeddings/ReshapeŠ 7functional_1/transformer_encoder/word_embeddings/GatherResourceGather@functional_1_transformer_encoder_word_embeddings_gather_resourceAfunctional_1/transformer_encoder/word_embeddings/Reshape:output:0* Tindices0*( _output_shapes : ’’’’’’’’’€* dtype029 7functional_1/transformer_encoder/word_embeddings/Gather÷ 9functional_1/transformer_encoder/word_embeddings/IdentityIdentity@functional_1/transformer_encoder/word_embeddings/Gather:output:0* T0*( _output_shapes : ’’’’’’’’’€2; 9functional_1/transformer_encoder/word_embeddings/Identity® 6functional_1/transformer_encoder/word_embeddings/ShapeShapeinput_word_ids* T0* _output_shapes :28 6functional_1/transformer_encoder/word_embeddings/ShapeĻ @functional_1/transformer_encoder/word_embeddings/concat/values_1Const* _output_shapes :* dtype0* valueB :€2B @functional_1/transformer_encoder/word_embeddings/concat/values_1¾ functional_1/transformer_encoder/type_embeddings/Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’2@ >functional_1/transformer_encoder/type_embeddings/Reshape/shape† 8functional_1/transformer_encoder/type_embeddings/ReshapeReshapeinput_type_idsGfunctional_1/transformer_encoder/type_embeddings/Reshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2: 8functional_1/transformer_encoder/type_embeddings/ReshapeĖ Afunctional_1/transformer_encoder/type_embeddings/one_hot/on_valueConst* _output_shapes :* dtype0* value B *€?2C Afunctional_1/transformer_encoder/type_embeddings/one_hot/on_valueĶ Bfunctional_1/transformer_encoder/type_embeddings/one_hot/off_valueConst* _output_shapes :* dtype0* value B *2D Bfunctional_1/transformer_encoder/type_embeddings/one_hot/off_valueĀ >functional_1/transformer_encoder/type_embeddings/one_hot/depthConst* _output_shapes :* dtype0* value B:2@ >functional_1/transformer_encoder/type_embeddings/one_hot/depthß 8functional_1/transformer_encoder/type_embeddings/one_hotOneHotAfunctional_1/transformer_encoder/type_embeddings/Reshape:output:0Gfunctional_1/transformer_encoder/type_embeddings/one_hot/depth:output:0Jfunctional_1/transformer_encoder/type_embeddings/one_hot/on_value:output:0Kfunctional_1/transformer_encoder/type_embeddings/one_hot/off_value:output:0* T0* TI0*' _output_shapes : ’’’’’’’’’2: 8functional_1/transformer_encoder/type_embeddings/one_hot” Ffunctional_1/transformer_encoder/type_embeddings/MatMul/ReadVariableOpReadVariableOpOfunctional_1_transformer_encoder_type_embeddings_matmul_readvariableop_resource* _output_shapes : €* dtype02H Ffunctional_1/transformer_encoder/type_embeddings/MatMul/ReadVariableOpĀ 7functional_1/transformer_encoder/type_embeddings/MatMulMatMulAfunctional_1/transformer_encoder/type_embeddings/one_hot:output:0Nfunctional_1/transformer_encoder/type_embeddings/MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€29 7functional_1/transformer_encoder/type_embeddings/MatMul® 6functional_1/transformer_encoder/type_embeddings/ShapeShapeinput_type_ids* T0* _output_shapes :28 6functional_1/transformer_encoder/type_embeddings/ShapeĻ @functional_1/transformer_encoder/type_embeddings/concat/values_1Const* _output_shapes :* dtype0* valueB :€2B @functional_1/transformer_encoder/type_embeddings/concat/values_1¾ functional_1/transformer_encoder/self_attention_mask/ones/LessLessCfunctional_1/transformer_encoder/self_attention_mask/ones/mul_1:z:0Ifunctional_1/transformer_encoder/self_attention_mask/ones/Less/y:output:0* T0* _output_shapes :2@ >functional_1/transformer_encoder/self_attention_mask/ones/LessŹ Bfunctional_1/transformer_encoder/self_attention_mask/ones/packed/2Const* _output_shapes :* dtype0* value B:2D Bfunctional_1/transformer_encoder/self_attention_mask/ones/packed/2£ @functional_1/transformer_encoder/self_attention_mask/ones/packedPackKfunctional_1/transformer_encoder/self_attention_mask/strided_slice:output:0Mfunctional_1/transformer_encoder/self_attention_mask/strided_slice_1:output:0Kfunctional_1/transformer_encoder/self_attention_mask/ones/packed/2:output:0* N* T0* _output_shapes :2B @functional_1/transformer_encoder/self_attention_mask/ones/packedĒ ?functional_1/transformer_encoder/self_attention_mask/ones/ConstConst* _output_shapes :* dtype0* value B *€?2A ?functional_1/transformer_encoder/self_attention_mask/ones/ConstŅ 9functional_1/transformer_encoder/self_attention_mask/onesFillIfunctional_1/transformer_encoder/self_attention_mask/ones/packed:output:0Hfunctional_1/transformer_encoder/self_attention_mask/ones/Const:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2; 9functional_1/transformer_encoder/self_attention_mask/onesĘ 8functional_1/transformer_encoder/self_attention_mask/mulMulBfunctional_1/transformer_encoder/self_attention_mask/ones:output:0=functional_1/transformer_encoder/self_attention_mask/Cast:y:0* T0*= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2: 8functional_1/transformer_encoder/self_attention_mask/mul… ffunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_0_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum/ReadVariableOpŃ Wfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/einsum/EinsumEinsum:functional_1/transformer_encoder/dropout/Identity:output:0nfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_0_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpĖ Ufunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/einsum/EinsumEinsum:functional_1/transformer_encoder/dropout/Identity:output:0lfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_0_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpŃ Wfunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/einsum/EinsumEinsum:functional_1/transformer_encoder/dropout/Identity:output:0nfunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_0_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_0/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_0/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_0/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_0/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_0/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_0/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_0/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_0/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_0/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_0/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_0/self_attention/Mul afunctional_1/transformer_encoder/transformer/layer_0/self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2c afunctional_1/transformer_encoder/transformer/layer_0/self_attention/masked_softmax/ExpandDims/dimĀ ]functional_1/transformer_encoder/transformer/layer_0/self_attention/masked_softmax/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_0/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_0/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_0_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/add¼ Gfunctional_1/transformer_encoder/transformer/layer_0/dropout_1/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_0/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_0/dropout_1/IdentityĖ 8functional_1/transformer_encoder/transformer/layer_0/addAddV2:functional_1/transformer_encoder/dropout/Identity:output:0Pfunctional_1/transformer_encoder/transformer/layer_0/dropout_1/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_0/addØ mfunctional_1/transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_0/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_0_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_0/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_0/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_0/intermediate/addÓ Efunctional_1/transformer_encoder/transformer/layer_0/activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2G Efunctional_1/transformer_encoder/transformer/layer_0/activation/Pow/yģ Cfunctional_1/transformer_encoder/transformer/layer_0/activation/PowPowIfunctional_1/transformer_encoder/transformer/layer_0/intermediate/add:z:0Nfunctional_1/transformer_encoder/transformer/layer_0/activation/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2E Cfunctional_1/transformer_encoder/transformer/layer_0/activation/PowÓ Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul/xConst* _output_shapes :* dtype0* value B *'7=2G Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul/xź Cfunctional_1/transformer_encoder/transformer/layer_0/activation/mulMulNfunctional_1/transformer_encoder/transformer/layer_0/activation/mul/x:output:0Gfunctional_1/transformer_encoder/transformer/layer_0/activation/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2E Cfunctional_1/transformer_encoder/transformer/layer_0/activation/mulē Cfunctional_1/transformer_encoder/transformer/layer_0/activation/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_0/intermediate/add:z:0Gfunctional_1/transformer_encoder/transformer/layer_0/activation/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2E Cfunctional_1/transformer_encoder/transformer/layer_0/activation/add× Gfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2I Gfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_1/xš Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul_1MulPfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_1/x:output:0Gfunctional_1/transformer_encoder/transformer/layer_0/activation/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul_1Ÿ Dfunctional_1/transformer_encoder/transformer/layer_0/activation/TanhTanhIfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2F Dfunctional_1/transformer_encoder/transformer/layer_0/activation/Tanh× Gfunctional_1/transformer_encoder/transformer/layer_0/activation/add_1/xConst* _output_shapes :* dtype0* value B *€?2I Gfunctional_1/transformer_encoder/transformer/layer_0/activation/add_1/xó Efunctional_1/transformer_encoder/transformer/layer_0/activation/add_1AddV2Pfunctional_1/transformer_encoder/transformer/layer_0/activation/add_1/x:output:0Hfunctional_1/transformer_encoder/transformer/layer_0/activation/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_0/activation/add_1× Gfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_2/xConst* _output_shapes :* dtype0* value B *?2I Gfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_2/xņ Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul_2MulPfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_2/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_0/activation/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul_2ė Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_0/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_0/activation/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_0/activation/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_0/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_0_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_0/output/einsum/Einsum/ReadVariableOp± Ifunctional_1/transformer_encoder/transformer/layer_0/output/einsum/EinsumEinsumIfunctional_1/transformer_encoder/transformer/layer_0/activation/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_0/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_0/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_0/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_0_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_0/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_0/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_0/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_0/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_0/output/add£ Gfunctional_1/transformer_encoder/transformer/layer_0/dropout_2/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_0/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_0/dropout_2/Identity÷ :functional_1/transformer_encoder/transformer/layer_0/add_1AddV2Pfunctional_1/transformer_encoder/transformer/layer_0/dropout_2/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_0/add_1˜ efunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_0/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_0/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_0/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_1_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_1_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_1_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_1/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_1/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_1/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_1/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_1/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_1/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_1/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_1/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_1/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_1/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_1/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_1/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_1/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_1_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/add¼ Gfunctional_1/transformer_encoder/transformer/layer_1/dropout_4/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_1/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_1/dropout_4/Identityė 8functional_1/transformer_encoder/transformer/layer_1/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Pfunctional_1/transformer_encoder/transformer/layer_1/dropout_4/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_1/addØ mfunctional_1/transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_1/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_1_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_1/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_1/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_1/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_1/activation_1/PowPowIfunctional_1/transformer_encoder/transformer/layer_1/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_1/activation_1/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_1/activation_1/Pow× Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_1/activation_1/mulMulPfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_1/activation_1/mulķ Efunctional_1/transformer_encoder/transformer/layer_1/activation_1/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_1/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_1/activation_1/addŪ Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_1/activation_1/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_1/activation_1/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_1/activation_1/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_1/activation_1/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_1/activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_1/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_1_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_1/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_1/activation_1/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_1/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_1/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_1_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_1/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_1/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_1/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_1/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_1/output/add£ Gfunctional_1/transformer_encoder/transformer/layer_1/dropout_5/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_1/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_1/dropout_5/Identity÷ :functional_1/transformer_encoder/transformer/layer_1/add_1AddV2Pfunctional_1/transformer_encoder/transformer/layer_1/dropout_5/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_1/add_1˜ efunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_1/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_1/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_1/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_2_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_2_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_2_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_2/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_2/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_2/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_2/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_2/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_2/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_2/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_2/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_2/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_2/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_2/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_2/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_2/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_2_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/add¼ Gfunctional_1/transformer_encoder/transformer/layer_2/dropout_7/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_2/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_2/dropout_7/Identityė 8functional_1/transformer_encoder/transformer/layer_2/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Pfunctional_1/transformer_encoder/transformer/layer_2/dropout_7/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_2/addØ mfunctional_1/transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_2/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_2_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_2/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_2/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_2/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_2/activation_2/PowPowIfunctional_1/transformer_encoder/transformer/layer_2/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_2/activation_2/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_2/activation_2/Pow× Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_2/activation_2/mulMulPfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_2/activation_2/mulķ Efunctional_1/transformer_encoder/transformer/layer_2/activation_2/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_2/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_2/activation_2/addŪ Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_2/activation_2/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_2/activation_2/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_2/activation_2/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_2/activation_2/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_2/activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_2/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_2_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_2/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_2/activation_2/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_2/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_2/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_2_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_2/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_2/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_2/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_2/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_2/output/add£ Gfunctional_1/transformer_encoder/transformer/layer_2/dropout_8/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_2/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_2/dropout_8/Identity÷ :functional_1/transformer_encoder/transformer/layer_2/add_1AddV2Pfunctional_1/transformer_encoder/transformer/layer_2/dropout_8/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_2/add_1˜ efunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_2/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_2/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_2/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_3_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_3_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_3_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_3/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_3/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_3/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_3/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_3/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_3/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_3/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_3/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_3/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_3/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_3/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_3/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_3/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_3_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_3/dropout_10/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_3/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_3/dropout_10/Identityģ 8functional_1/transformer_encoder/transformer/layer_3/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_3/dropout_10/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_3/addØ mfunctional_1/transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_3/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_3_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_3/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_3/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_3/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_3/activation_3/PowPowIfunctional_1/transformer_encoder/transformer/layer_3/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_3/activation_3/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_3/activation_3/Pow× Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_3/activation_3/mulMulPfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_3/activation_3/mulķ Efunctional_1/transformer_encoder/transformer/layer_3/activation_3/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_3/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_3/activation_3/addŪ Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_3/activation_3/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_3/activation_3/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_3/activation_3/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_3/activation_3/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_3/activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_3/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_3_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_3/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_3/activation_3/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_3/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_3/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_3_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_3/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_3/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_3/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_3/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_3/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_3/dropout_11/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_3/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_3/dropout_11/Identityų :functional_1/transformer_encoder/transformer/layer_3/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_3/dropout_11/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_3/add_1˜ efunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_3/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_3/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_3/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_4_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_4_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_4_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_4/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_4/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_4/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_4/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_4/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_4/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_4/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_4/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_4/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_4/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_4/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_4/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_4/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_4_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_4/dropout_13/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_4/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_4/dropout_13/Identityģ 8functional_1/transformer_encoder/transformer/layer_4/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_4/dropout_13/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_4/addØ mfunctional_1/transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_4/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_4_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_4/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_4/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_4/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_4/activation_4/PowPowIfunctional_1/transformer_encoder/transformer/layer_4/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_4/activation_4/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_4/activation_4/Pow× Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_4/activation_4/mulMulPfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_4/activation_4/mulķ Efunctional_1/transformer_encoder/transformer/layer_4/activation_4/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_4/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_4/activation_4/addŪ Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_4/activation_4/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_4/activation_4/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_4/activation_4/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_4/activation_4/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_4/activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_4/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_4_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_4/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_4/activation_4/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_4/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_4/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_4_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_4/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_4/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_4/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_4/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_4/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_4/dropout_14/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_4/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_4/dropout_14/Identityų :functional_1/transformer_encoder/transformer/layer_4/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_4/dropout_14/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_4/add_1˜ efunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_4/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_4/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_4/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_5_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_5_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_5_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_5/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_5/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_5/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_5/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_5/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_5/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_5/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_5/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_5/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_5/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_5/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_5/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_5/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_5_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_5/dropout_16/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_5/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_5/dropout_16/Identityģ 8functional_1/transformer_encoder/transformer/layer_5/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_5/dropout_16/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_5/addØ mfunctional_1/transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_5/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_5_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_5/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_5/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_5/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_5/activation_5/PowPowIfunctional_1/transformer_encoder/transformer/layer_5/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_5/activation_5/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_5/activation_5/Pow× Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_5/activation_5/mulMulPfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_5/activation_5/mulķ Efunctional_1/transformer_encoder/transformer/layer_5/activation_5/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_5/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_5/activation_5/addŪ Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_5/activation_5/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_5/activation_5/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_5/activation_5/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_5/activation_5/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_5/activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_5/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_5_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_5/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_5/activation_5/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_5/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_5/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_5_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_5/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_5/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_5/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_5/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_5/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_5/dropout_17/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_5/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_5/dropout_17/Identityų :functional_1/transformer_encoder/transformer/layer_5/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_5/dropout_17/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_5/add_1˜ efunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_5/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_5/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_5/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_6_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_6_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_6_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_6/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_6/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_6/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_6/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_6/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_6/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_6/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_6/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_6/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_6/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_6/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_6/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_6/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_6_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_6/dropout_19/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_6/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_6/dropout_19/Identityģ 8functional_1/transformer_encoder/transformer/layer_6/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_6/dropout_19/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_6/addØ mfunctional_1/transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_6/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_6_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_6/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_6/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_6/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_6/activation_6/PowPowIfunctional_1/transformer_encoder/transformer/layer_6/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_6/activation_6/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_6/activation_6/Pow× Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_6/activation_6/mulMulPfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_6/activation_6/mulķ Efunctional_1/transformer_encoder/transformer/layer_6/activation_6/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_6/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_6/activation_6/addŪ Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_6/activation_6/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_6/activation_6/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_6/activation_6/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_6/activation_6/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_6/activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_6/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_6_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_6/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_6/activation_6/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_6/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_6/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_6_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_6/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_6/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_6/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_6/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_6/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_6/dropout_20/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_6/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_6/dropout_20/Identityų :functional_1/transformer_encoder/transformer/layer_6/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_6/dropout_20/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_6/add_1˜ efunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_6/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_6/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_6/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_7_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_7_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_7_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_7/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_7/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_7/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_7/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_7/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_7/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_7/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_7/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_7/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_7/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_7/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_7/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_7/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_7_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_7/dropout_22/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_7/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_7/dropout_22/Identityģ 8functional_1/transformer_encoder/transformer/layer_7/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_7/dropout_22/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_7/addØ mfunctional_1/transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_7/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_7_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_7/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_7/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_7/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_7/activation_7/PowPowIfunctional_1/transformer_encoder/transformer/layer_7/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_7/activation_7/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_7/activation_7/Pow× Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_7/activation_7/mulMulPfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_7/activation_7/mulķ Efunctional_1/transformer_encoder/transformer/layer_7/activation_7/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_7/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_7/activation_7/addŪ Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_7/activation_7/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_7/activation_7/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_7/activation_7/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_7/activation_7/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_7/activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_7/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_7_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_7/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_7/activation_7/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_7/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_7/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_7_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_7/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_7/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_7/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_7/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_7/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_7/dropout_23/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_7/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_7/dropout_23/Identityų :functional_1/transformer_encoder/transformer/layer_7/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_7/dropout_23/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_7/add_1˜ efunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_7/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_7/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_7/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_8_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_8_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_8_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_8/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_8/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_8/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_8/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_8/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_8/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_8/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_8/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_8/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_8/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_8/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_8/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_8/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_8_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_8/dropout_25/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_8/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_8/dropout_25/Identityģ 8functional_1/transformer_encoder/transformer/layer_8/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_8/dropout_25/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_8/addØ mfunctional_1/transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_8/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_8_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_8/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_8/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_8/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_8/activation_8/PowPowIfunctional_1/transformer_encoder/transformer/layer_8/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_8/activation_8/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_8/activation_8/Pow× Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_8/activation_8/mulMulPfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_8/activation_8/mulķ Efunctional_1/transformer_encoder/transformer/layer_8/activation_8/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_8/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_8/activation_8/addŪ Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_8/activation_8/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_8/activation_8/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_8/activation_8/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_8/activation_8/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_8/activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_8/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_8_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_8/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_8/activation_8/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_8/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_8/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_8_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_8/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_8/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_8/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_8/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_8/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_8/dropout_26/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_8/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_8/dropout_26/Identityų :functional_1/transformer_encoder/transformer/layer_8/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_8/dropout_26/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_8/add_1˜ efunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_8/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_8/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_8/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1… ffunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_9_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/addAddV2`functional_1/transformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/add’ dfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02f dfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpė Ufunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0lfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2W Ufunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/einsum/EinsumÜ Zfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOpReadVariableOpcfunctional_1_transformer_encoder_transformer_layer_9_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02\ Zfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOpŖ Kfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/addAddV2^functional_1/transformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum:output:0bfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2M Kfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/add… ffunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpofunctional_1_transformer_encoder_transformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02h ffunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpń Wfunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Y Wfunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsumā \functional_1/transformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOpReadVariableOpefunctional_1_transformer_encoder_transformer_layer_9_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02^ \functional_1/transformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOp² Mfunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/addAddV2`functional_1/transformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum:output:0dfunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2O Mfunctional_1/transformer_encoder/transformer/layer_9/self_attention/value/addČ Qfunctional_1/transformer_encoder/transformer/layer_9/self_attention/einsum/EinsumEinsumOfunctional_1/transformer_encoder/transformer/layer_9/self_attention/key/add:z:0Qfunctional_1/transformer_encoder/transformer/layer_9/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2S Qfunctional_1/transformer_encoder/transformer/layer_9/self_attention/einsum/EinsumŪ Ifunctional_1/transformer_encoder/transformer/layer_9/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2K Ifunctional_1/transformer_encoder/transformer/layer_9/self_attention/Mul/y• Gfunctional_1/transformer_encoder/transformer/layer_9/self_attention/MulMulZfunctional_1/transformer_encoder/transformer/layer_9/self_attention/einsum/Einsum:output:0Rfunctional_1/transformer_encoder/transformer/layer_9/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2I Gfunctional_1/transformer_encoder/transformer/layer_9/self_attention/Mul” cfunctional_1/transformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2e cfunctional_1/transformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dimČ _functional_1/transformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/ExpandDims ExpandDimsabcd2U Sfunctional_1/transformer_encoder/transformer/layer_9/self_attention/einsum_1/Einsum¦ qfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpzfunctional_1_transformer_encoder_transformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02s qfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ bfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/EinsumEinsum\functional_1/transformer_encoder/transformer/layer_9/self_attention/einsum_1/Einsum:output:0yfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2d bfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum€ gfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_9_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02i gfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOpŪ Xfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/addAddV2kfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum:output:0ofunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Z Xfunctional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/add¾ Hfunctional_1/transformer_encoder/transformer/layer_9/dropout_28/IdentityIdentity\functional_1/transformer_encoder/transformer/layer_9/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_9/dropout_28/Identityģ 8functional_1/transformer_encoder/transformer/layer_9/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Qfunctional_1/transformer_encoder/transformer/layer_9/dropout_28/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8functional_1/transformer_encoder/transformer/layer_9/addØ mfunctional_1/transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2o mfunctional_1/transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesČ [functional_1/transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/meanMeanabd2Q Ofunctional_1/transformer_encoder/transformer/layer_9/intermediate/einsum/EinsumĒ Tfunctional_1/transformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOpReadVariableOp]functional_1_transformer_encoder_transformer_layer_9_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02V Tfunctional_1/transformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOp Efunctional_1/transformer_encoder/transformer/layer_9/intermediate/addAddV2Xfunctional_1/transformer_encoder/transformer/layer_9/intermediate/einsum/Einsum:output:0\functional_1/transformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_9/intermediate/add× Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2I Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/Pow/yņ Efunctional_1/transformer_encoder/transformer/layer_9/activation_9/PowPowIfunctional_1/transformer_encoder/transformer/layer_9/intermediate/add:z:0Pfunctional_1/transformer_encoder/transformer/layer_9/activation_9/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_9/activation_9/Pow× Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul/xConst* _output_shapes :* dtype0* value B *'7=2I Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul/xņ Efunctional_1/transformer_encoder/transformer/layer_9/activation_9/mulMulPfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_9/activation_9/mulķ Efunctional_1/transformer_encoder/transformer/layer_9/activation_9/addAddV2Ifunctional_1/transformer_encoder/transformer/layer_9/intermediate/add:z:0Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2G Efunctional_1/transformer_encoder/transformer/layer_9/activation_9/addŪ Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2K Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_1/xų Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_1MulRfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_1/x:output:0Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_1„ Ffunctional_1/transformer_encoder/transformer/layer_9/activation_9/TanhTanhKfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_9/activation_9/TanhŪ Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/add_1/xConst* _output_shapes :* dtype0* value B *€?2K Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/add_1/xū Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_9/activation_9/add_1/x:output:0Jfunctional_1/transformer_encoder/transformer/layer_9/activation_9/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/add_1Ū Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_2/xConst* _output_shapes :* dtype0* value B *?2K Ifunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_2/xś Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_2MulRfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_2/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_9/activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_2ń Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_3MulIfunctional_1/transformer_encoder/transformer/layer_9/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_3Ų Xfunctional_1/transformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOpReadVariableOpafunctional_1_transformer_encoder_transformer_layer_9_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02Z Xfunctional_1/transformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOp³ Ifunctional_1/transformer_encoder/transformer/layer_9/output/einsum/EinsumEinsumKfunctional_1/transformer_encoder/transformer/layer_9/activation_9/mul_3:z:0`functional_1/transformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2K Ifunctional_1/transformer_encoder/transformer/layer_9/output/einsum/Einsumµ Nfunctional_1/transformer_encoder/transformer/layer_9/output/add/ReadVariableOpReadVariableOpWfunctional_1_transformer_encoder_transformer_layer_9_output_add_readvariableop_resource* _output_shapes :€* dtype02P Nfunctional_1/transformer_encoder/transformer/layer_9/output/add/ReadVariableOp÷ ?functional_1/transformer_encoder/transformer/layer_9/output/addAddV2Rfunctional_1/transformer_encoder/transformer/layer_9/output/einsum/Einsum:output:0Vfunctional_1/transformer_encoder/transformer/layer_9/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?functional_1/transformer_encoder/transformer/layer_9/output/add„ Hfunctional_1/transformer_encoder/transformer/layer_9/dropout_29/IdentityIdentityCfunctional_1/transformer_encoder/transformer/layer_9/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_9/dropout_29/Identityų :functional_1/transformer_encoder/transformer/layer_9/add_1AddV2Qfunctional_1/transformer_encoder/transformer/layer_9/dropout_29/Identity:output:0bfunctional_1/transformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :functional_1/transformer_encoder/transformer/layer_9/add_1˜ efunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/mean/reduction_indices² Sfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/meanMean>functional_1/transformer_encoder/transformer/layer_9/add_1:z:0nfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Sfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/meanē [functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/StopGradient StopGradient\functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2] [functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/StopGradientæ `functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/SquaredDifferenceSquaredDifference>functional_1/transformer_encoder/transformer/layer_9/add_1:z:0dfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2b `functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/SquaredDifference  ifunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2k ifunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/variance/reduction_indicesä Wfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/varianceMeandfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/SquaredDifference:z:0rfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Y Wfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/varianceõ Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2X Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add/y· Tfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/addAddV2`functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/variance:output:0_functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2V Tfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/addŅ Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/RsqrtRsqrtXfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/Rsqrtō cfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOplfunctional_1_transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02e cfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOp¼ Tfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mulMulZfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/Rsqrt:y:0kfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul‘ Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_1Mul>functional_1/transformer_encoder/transformer/layer_9/add_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_1Æ Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_2Mul\functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/moments/mean:output:0Xfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_2č _functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02a _functional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOpø Tfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/subSubgfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp:value:0Zfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Tfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/subÆ Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1AddV2Zfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_1:z:0Xfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2X Vfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1ˆ gfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02i gfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpō Xfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0ofunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Z Xfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsumå ]functional_1/transformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOpReadVariableOpffunctional_1_transformer_encoder_transformer_layer_10_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02_ ]functional_1/transformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOp¶ Nfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/addAddV2afunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum:output:0efunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2P Nfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/add‚ efunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpnfunctional_1_transformer_encoder_transformer_layer_10_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02g efunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOpī Vfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0mfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2X Vfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsumß [functional_1/transformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOpReadVariableOpdfunctional_1_transformer_encoder_transformer_layer_10_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02] [functional_1/transformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOp® Lfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/addAddV2_functional_1/transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum:output:0cfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2N Lfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/addˆ gfunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02i gfunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpō Xfunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/einsum/EinsumEinsumZfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0ofunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Z Xfunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsumå ]functional_1/transformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOpReadVariableOpffunctional_1_transformer_encoder_transformer_layer_10_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02_ ]functional_1/transformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOp¶ Nfunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/addAddV2afunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum:output:0efunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2P Nfunctional_1/transformer_encoder/transformer/layer_10/self_attention/value/addĢ Rfunctional_1/transformer_encoder/transformer/layer_10/self_attention/einsum/EinsumEinsumPfunctional_1/transformer_encoder/transformer/layer_10/self_attention/key/add:z:0Rfunctional_1/transformer_encoder/transformer/layer_10/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2T Rfunctional_1/transformer_encoder/transformer/layer_10/self_attention/einsum/EinsumŻ Jfunctional_1/transformer_encoder/transformer/layer_10/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2L Jfunctional_1/transformer_encoder/transformer/layer_10/self_attention/Mul/y™ Hfunctional_1/transformer_encoder/transformer/layer_10/self_attention/MulMul[functional_1/transformer_encoder/transformer/layer_10/self_attention/einsum/Einsum:output:0Sfunctional_1/transformer_encoder/transformer/layer_10/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2J Hfunctional_1/transformer_encoder/transformer/layer_10/self_attention/Mul˜ efunctional_1/transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimĪ afunctional_1/transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims ExpandDimsabcd2V Tfunctional_1/transformer_encoder/transformer/layer_10/self_attention/einsum_1/Einsum© rfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOp{functional_1_transformer_encoder_transformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02t rfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOp• cfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/EinsumEinsum]functional_1/transformer_encoder/transformer/layer_10/self_attention/einsum_1/Einsum:output:0zfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2e cfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsumƒ hfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOpReadVariableOpqfunctional_1_transformer_encoder_transformer_layer_10_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02j hfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOpß Yfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/addAddV2lfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum:output:0pfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2[ Yfunctional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/addĮ Ifunctional_1/transformer_encoder/transformer/layer_10/dropout_31/IdentityIdentity]functional_1/transformer_encoder/transformer/layer_10/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_10/dropout_31/Identityļ 9functional_1/transformer_encoder/transformer/layer_10/addAddV2Zfunctional_1/transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0Rfunctional_1/transformer_encoder/transformer/layer_10/dropout_31/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9functional_1/transformer_encoder/transformer/layer_10/addŖ nfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2p nfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean/reduction_indicesĢ \functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/meanMean=functional_1/transformer_encoder/transformer/layer_10/add:z:0wfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2^ \functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean‚ dfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/StopGradient StopGradientefunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2f dfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/StopGradientŁ ifunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference=functional_1/transformer_encoder/transformer/layer_10/add:z:0mfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2k ifunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/SquaredDifference² rfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2t rfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance/reduction_indicesˆ `functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/varianceMeanmfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/SquaredDifference:z:0{functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2b `functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance‡ _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2a _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add/yŪ ]functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/addAddV2ifunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance:output:0hfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2_ ]functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/addķ _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/RsqrtRsqrtafunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2a _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/Rsqrt lfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpufunctional_1_transformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02n lfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpą ]functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mulMulcfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/Rsqrt:y:0tfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2_ ]functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul« _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Mul=functional_1/transformer_encoder/transformer/layer_10/add:z:0afunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2a _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Ó _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2Mulefunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean:output:0afunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2a _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2ƒ hfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpqfunctional_1_transformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02j hfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpÜ ]functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/subSubpfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0cfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2_ ]functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/subÓ _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1AddV2cfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1:z:0afunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2a _functional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1ķ _functional_1/transformer_encoder/transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_10_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02a _functional_1/transformer_encoder/transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpą Pfunctional_1/transformer_encoder/transformer/layer_10/intermediate/einsum/EinsumEinsumcfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0gfunctional_1/transformer_encoder/transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2R Pfunctional_1/transformer_encoder/transformer/layer_10/intermediate/einsum/EinsumŹ Ufunctional_1/transformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOpReadVariableOp^functional_1_transformer_encoder_transformer_layer_10_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02W Ufunctional_1/transformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOp“ Ffunctional_1/transformer_encoder/transformer/layer_10/intermediate/addAddV2Yfunctional_1/transformer_encoder/transformer/layer_10/intermediate/einsum/Einsum:output:0]functional_1/transformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_10/intermediate/addŪ Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/Pow/yConst* _output_shapes :* dtype0* value B *@@2K Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/Pow/ył Gfunctional_1/transformer_encoder/transformer/layer_10/activation_10/PowPowJfunctional_1/transformer_encoder/transformer/layer_10/intermediate/add:z:0Rfunctional_1/transformer_encoder/transformer/layer_10/activation_10/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_10/activation_10/PowŪ Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul/xConst* _output_shapes :* dtype0* value B *'7=2K Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul/xś Gfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mulMulRfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mulō Gfunctional_1/transformer_encoder/transformer/layer_10/activation_10/addAddV2Jfunctional_1/transformer_encoder/transformer/layer_10/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_10/activation_10/addß Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2M Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_1/x€ Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_1MulTfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_1/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_1« Hfunctional_1/transformer_encoder/transformer/layer_10/activation_10/TanhTanhMfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_10/activation_10/Tanhß Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2M Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/add_1/xƒ Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/add_1AddV2Tfunctional_1/transformer_encoder/transformer/layer_10/activation_10/add_1/x:output:0Lfunctional_1/transformer_encoder/transformer/layer_10/activation_10/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/add_1ß Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2M Kfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_2/x‚ Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_2MulTfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_2/x:output:0Mfunctional_1/transformer_encoder/transformer/layer_10/activation_10/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_2ų Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_3MulJfunctional_1/transformer_encoder/transformer/layer_10/intermediate/add:z:0Mfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_3Ū Yfunctional_1/transformer_encoder/transformer/layer_10/output/einsum/Einsum/ReadVariableOpReadVariableOpbfunctional_1_transformer_encoder_transformer_layer_10_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02[ Yfunctional_1/transformer_encoder/transformer/layer_10/output/einsum/Einsum/ReadVariableOpø Jfunctional_1/transformer_encoder/transformer/layer_10/output/einsum/EinsumEinsumMfunctional_1/transformer_encoder/transformer/layer_10/activation_10/mul_3:z:0afunctional_1/transformer_encoder/transformer/layer_10/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2L Jfunctional_1/transformer_encoder/transformer/layer_10/output/einsum/Einsumø Ofunctional_1/transformer_encoder/transformer/layer_10/output/add/ReadVariableOpReadVariableOpXfunctional_1_transformer_encoder_transformer_layer_10_output_add_readvariableop_resource* _output_shapes :€* dtype02Q Ofunctional_1/transformer_encoder/transformer/layer_10/output/add/ReadVariableOpū @functional_1/transformer_encoder/transformer/layer_10/output/addAddV2Sfunctional_1/transformer_encoder/transformer/layer_10/output/einsum/Einsum:output:0Wfunctional_1/transformer_encoder/transformer/layer_10/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @functional_1/transformer_encoder/transformer/layer_10/output/addØ Ifunctional_1/transformer_encoder/transformer/layer_10/dropout_32/IdentityIdentityDfunctional_1/transformer_encoder/transformer/layer_10/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_10/dropout_32/Identityü ;functional_1/transformer_encoder/transformer/layer_10/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_10/dropout_32/Identity:output:0cfunctional_1/transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;functional_1/transformer_encoder/transformer/layer_10/add_1š ffunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2h ffunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/mean/reduction_indices¶ Tfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/meanMean?functional_1/transformer_encoder/transformer/layer_10/add_1:z:0ofunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2V Tfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/meanź \functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/StopGradient StopGradient]functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2^ \functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/StopGradientĆ afunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/SquaredDifferenceSquaredDifference?functional_1/transformer_encoder/transformer/layer_10/add_1:z:0efunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2c afunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/SquaredDifference¢ jfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2l jfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance/reduction_indicesč Xfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/varianceMeanefunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/SquaredDifference:z:0sfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Z Xfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance÷ Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2Y Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add/y» Ufunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/addAddV2afunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance:output:0`functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2W Ufunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/addÕ Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/RsqrtRsqrtYfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Y Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/Rsqrt÷ dfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02f dfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul/ReadVariableOpĄ Ufunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mulMul[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/Rsqrt:y:0lfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2W Ufunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul• Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_1Mul?functional_1/transformer_encoder/transformer/layer_10/add_1:z:0Yfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Y Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_1³ Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_2Mul]functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/moments/mean:output:0Yfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Y Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_2ė `functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpifunctional_1_transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02b `functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/ReadVariableOp¼ Ufunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/subSubhfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/ReadVariableOp:value:0[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2W Ufunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/sub³ Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1AddV2[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_1:z:0Yfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Y Wfunctional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1ˆ gfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_11_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02i gfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum/ReadVariableOpõ Xfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/einsum/EinsumEinsum[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0ofunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Z Xfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsumå ]functional_1/transformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOpReadVariableOpffunctional_1_transformer_encoder_transformer_layer_11_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02_ ]functional_1/transformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOp¶ Nfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/addAddV2afunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum:output:0efunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2P Nfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/add‚ efunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpnfunctional_1_transformer_encoder_transformer_layer_11_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02g efunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOpļ Vfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/einsum/EinsumEinsum[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0mfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2X Vfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsumß [functional_1/transformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOpReadVariableOpdfunctional_1_transformer_encoder_transformer_layer_11_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02] [functional_1/transformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOp® Lfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/addAddV2_functional_1/transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum:output:0cfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2N Lfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/addˆ gfunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOppfunctional_1_transformer_encoder_transformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02i gfunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpõ Xfunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/einsum/EinsumEinsum[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0ofunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2Z Xfunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsumå ]functional_1/transformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOpReadVariableOpffunctional_1_transformer_encoder_transformer_layer_11_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02_ ]functional_1/transformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOp¶ Nfunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/addAddV2afunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum:output:0efunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2P Nfunctional_1/transformer_encoder/transformer/layer_11/self_attention/value/addĢ Rfunctional_1/transformer_encoder/transformer/layer_11/self_attention/einsum/EinsumEinsumPfunctional_1/transformer_encoder/transformer/layer_11/self_attention/key/add:z:0Rfunctional_1/transformer_encoder/transformer/layer_11/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2T Rfunctional_1/transformer_encoder/transformer/layer_11/self_attention/einsum/EinsumŻ Jfunctional_1/transformer_encoder/transformer/layer_11/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2L Jfunctional_1/transformer_encoder/transformer/layer_11/self_attention/Mul/y™ Hfunctional_1/transformer_encoder/transformer/layer_11/self_attention/MulMul[functional_1/transformer_encoder/transformer/layer_11/self_attention/einsum/Einsum:output:0Sfunctional_1/transformer_encoder/transformer/layer_11/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2J Hfunctional_1/transformer_encoder/transformer/layer_11/self_attention/Mul˜ efunctional_1/transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2g efunctional_1/transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimĪ afunctional_1/transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims ExpandDimsabcd2V Tfunctional_1/transformer_encoder/transformer/layer_11/self_attention/einsum_1/Einsum© rfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOp{functional_1_transformer_encoder_transformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02t rfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOp• cfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/EinsumEinsum]functional_1/transformer_encoder/transformer/layer_11/self_attention/einsum_1/Einsum:output:0zfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2e cfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsumƒ hfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOpReadVariableOpqfunctional_1_transformer_encoder_transformer_layer_11_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02j hfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOpß Yfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/addAddV2lfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum:output:0pfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2[ Yfunctional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/addĮ Ifunctional_1/transformer_encoder/transformer/layer_11/dropout_34/IdentityIdentity]functional_1/transformer_encoder/transformer/layer_11/self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_11/dropout_34/Identityš 9functional_1/transformer_encoder/transformer/layer_11/addAddV2[functional_1/transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0Rfunctional_1/transformer_encoder/transformer/layer_11/dropout_34/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9functional_1/transformer_encoder/transformer/layer_11/addŖ nfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2p nfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean/reduction_indicesĢ \functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/meanMean=functional_1/transformer_encoder/transformer/layer_11/add:z:0wfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2^ \functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean‚ dfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/StopGradient StopGradientefunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2f dfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/StopGradientŁ ifunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference=functional_1/transformer_encoder/transformer/layer_11/add:z:0mfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2k ifunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/SquaredDifference² rfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2t rfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance/reduction_indicesˆ `functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/varianceMeanmfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/SquaredDifference:z:0{functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2b `functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance‡ _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2a _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add/yŪ ]functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/addAddV2ifunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance:output:0hfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2_ ]functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/addķ _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/RsqrtRsqrtafunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2a _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/Rsqrt lfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpufunctional_1_transformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02n lfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpą ]functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mulMulcfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/Rsqrt:y:0tfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2_ ]functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul« _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Mul=functional_1/transformer_encoder/transformer/layer_11/add:z:0afunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2a _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Ó _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2Mulefunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean:output:0afunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2a _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2ƒ hfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpqfunctional_1_transformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02j hfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpÜ ]functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/subSubpfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0cfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2_ ]functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/subÓ _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1AddV2cfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1:z:0afunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2a _functional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1ķ _functional_1/transformer_encoder/transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpReadVariableOphfunctional_1_transformer_encoder_transformer_layer_11_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02a _functional_1/transformer_encoder/transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpą Pfunctional_1/transformer_encoder/transformer/layer_11/intermediate/einsum/EinsumEinsumcfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0gfunctional_1/transformer_encoder/transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2R Pfunctional_1/transformer_encoder/transformer/layer_11/intermediate/einsum/EinsumŹ Ufunctional_1/transformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOpReadVariableOp^functional_1_transformer_encoder_transformer_layer_11_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02W Ufunctional_1/transformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOp“ Ffunctional_1/transformer_encoder/transformer/layer_11/intermediate/addAddV2Yfunctional_1/transformer_encoder/transformer/layer_11/intermediate/einsum/Einsum:output:0]functional_1/transformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ffunctional_1/transformer_encoder/transformer/layer_11/intermediate/addŪ Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/Pow/yConst* _output_shapes :* dtype0* value B *@@2K Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/Pow/ył Gfunctional_1/transformer_encoder/transformer/layer_11/activation_11/PowPowJfunctional_1/transformer_encoder/transformer/layer_11/intermediate/add:z:0Rfunctional_1/transformer_encoder/transformer/layer_11/activation_11/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_11/activation_11/PowŪ Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul/xConst* _output_shapes :* dtype0* value B *'7=2K Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul/xś Gfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mulMulRfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mulō Gfunctional_1/transformer_encoder/transformer/layer_11/activation_11/addAddV2Jfunctional_1/transformer_encoder/transformer/layer_11/intermediate/add:z:0Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gfunctional_1/transformer_encoder/transformer/layer_11/activation_11/addß Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2M Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_1/x€ Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_1MulTfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_1/x:output:0Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_1« Hfunctional_1/transformer_encoder/transformer/layer_11/activation_11/TanhTanhMfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Hfunctional_1/transformer_encoder/transformer/layer_11/activation_11/Tanhß Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2M Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/add_1/xƒ Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/add_1AddV2Tfunctional_1/transformer_encoder/transformer/layer_11/activation_11/add_1/x:output:0Lfunctional_1/transformer_encoder/transformer/layer_11/activation_11/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/add_1ß Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2M Kfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_2/x‚ Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_2MulTfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_2/x:output:0Mfunctional_1/transformer_encoder/transformer/layer_11/activation_11/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_2ų Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_3MulJfunctional_1/transformer_encoder/transformer/layer_11/intermediate/add:z:0Mfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_3Ū Yfunctional_1/transformer_encoder/transformer/layer_11/output/einsum/Einsum/ReadVariableOpReadVariableOpbfunctional_1_transformer_encoder_transformer_layer_11_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02[ Yfunctional_1/transformer_encoder/transformer/layer_11/output/einsum/Einsum/ReadVariableOpø Jfunctional_1/transformer_encoder/transformer/layer_11/output/einsum/EinsumEinsumMfunctional_1/transformer_encoder/transformer/layer_11/activation_11/mul_3:z:0afunctional_1/transformer_encoder/transformer/layer_11/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2L Jfunctional_1/transformer_encoder/transformer/layer_11/output/einsum/Einsumø Ofunctional_1/transformer_encoder/transformer/layer_11/output/add/ReadVariableOpReadVariableOpXfunctional_1_transformer_encoder_transformer_layer_11_output_add_readvariableop_resource* _output_shapes :€* dtype02Q Ofunctional_1/transformer_encoder/transformer/layer_11/output/add/ReadVariableOpū @functional_1/transformer_encoder/transformer/layer_11/output/addAddV2Sfunctional_1/transformer_encoder/transformer/layer_11/output/einsum/Einsum:output:0Wfunctional_1/transformer_encoder/transformer/layer_11/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @functional_1/transformer_encoder/transformer/layer_11/output/addØ Ifunctional_1/transformer_encoder/transformer/layer_11/dropout_35/IdentityIdentityDfunctional_1/transformer_encoder/transformer/layer_11/output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Ifunctional_1/transformer_encoder/transformer/layer_11/dropout_35/Identityü ;functional_1/transformer_encoder/transformer/layer_11/add_1AddV2Rfunctional_1/transformer_encoder/transformer/layer_11/dropout_35/Identity:output:0cfunctional_1/transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;functional_1/transformer_encoder/transformer/layer_11/add_1š ffunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2h ffunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/mean/reduction_indices¶ Tfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/meanMean?functional_1/transformer_encoder/transformer/layer_11/add_1:z:0ofunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2V Tfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/meanź \functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/StopGradient StopGradient]functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2^ \functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/StopGradientĆ afunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/SquaredDifferenceSquaredDifference?functional_1/transformer_encoder/transformer/layer_11/add_1:z:0efunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2c afunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/SquaredDifference¢ jfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2l jfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance/reduction_indicesč Xfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/varianceMeanefunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/SquaredDifference:z:0sfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Z Xfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance÷ Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2Y Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add/y» Ufunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/addAddV2afunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance:output:0`functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2W Ufunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/addÕ Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/RsqrtRsqrtYfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Y Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/Rsqrt÷ dfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpmfunctional_1_transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02f dfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul/ReadVariableOpĄ Ufunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mulMul[functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/Rsqrt:y:0lfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2W Ufunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul• Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_1Mul?functional_1/transformer_encoder/transformer/layer_11/add_1:z:0Yfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Y Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_1³ Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_2Mul]functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/moments/mean:output:0Yfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Y Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_2ė `functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpifunctional_1_transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02b `functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/ReadVariableOp¼ Ufunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/subSubhfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/ReadVariableOp:value:0[functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2W Ufunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/sub³ Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1AddV2[functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_1:z:0Yfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Y Wfunctional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1Ļ ;functional_1/transformer_encoder/lambda/strided_slice/stackConst* _output_shapes :* dtype0*! valueB" 2= ;functional_1/transformer_encoder/lambda/strided_slice/stackÓ =functional_1/transformer_encoder/lambda/strided_slice/stack_1Const* _output_shapes :* dtype0*! valueB" 2? =functional_1/transformer_encoder/lambda/strided_slice/stack_1Ó =functional_1/transformer_encoder/lambda/strided_slice/stack_2Const* _output_shapes :* dtype0*! valueB" 2? =functional_1/transformer_encoder/lambda/strided_slice/stack_2  5functional_1/transformer_encoder/lambda/strided_slice StridedSlice[functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1:z:0Dfunctional_1/transformer_encoder/lambda/strided_slice/stack:output:0Ffunctional_1/transformer_encoder/lambda/strided_slice/stack_1:output:0Ffunctional_1/transformer_encoder/lambda/strided_slice/stack_2:output:0* Index0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* begin_mask* end_mask27 5functional_1/transformer_encoder/lambda/strided_slice÷ /functional_1/transformer_encoder/lambda/SqueezeSqueeze>functional_1/transformer_encoder/lambda/strided_slice:output:0* T0*( _output_shapes : ’’’’’’’’’€* squeeze_dims 21 /functional_1/transformer_encoder/lambda/Squeeze„ Gfunctional_1/transformer_encoder/pooler_transform/MatMul/ReadVariableOpReadVariableOpPfunctional_1_transformer_encoder_pooler_transform_matmul_readvariableop_resource* _output_shapes : €€* dtype02I Gfunctional_1/transformer_encoder/pooler_transform/MatMul/ReadVariableOp¼ 8functional_1/transformer_encoder/pooler_transform/MatMulMatMul8functional_1/transformer_encoder/lambda/Squeeze:output:0Ofunctional_1/transformer_encoder/pooler_transform/MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2: 8functional_1/transformer_encoder/pooler_transform/MatMul£ Hfunctional_1/transformer_encoder/pooler_transform/BiasAdd/ReadVariableOpReadVariableOpQfunctional_1_transformer_encoder_pooler_transform_biasadd_readvariableop_resource* _output_shapes :€* dtype02J Hfunctional_1/transformer_encoder/pooler_transform/BiasAdd/ReadVariableOpŹ 9functional_1/transformer_encoder/pooler_transform/BiasAddBiasAddBfunctional_1/transformer_encoder/pooler_transform/MatMul:product:0Pfunctional_1/transformer_encoder/pooler_transform/BiasAdd/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2; 9functional_1/transformer_encoder/pooler_transform/BiasAddļ 6functional_1/transformer_encoder/pooler_transform/TanhTanhBfunctional_1/transformer_encoder/pooler_transform/BiasAdd:output:0* T0*( _output_shapes : ’’’’’’’’’€28 6functional_1/transformer_encoder/pooler_transform/Tanh IdentityIdentity:functional_1/transformer_encoder/pooler_transform/Tanh:y:0* T0*( _output_shapes : ’’’’’’’’’€2 IdentityĮ Identity_1Identity[functional_1/transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : Ž ] A__inference_lambda_layer_call_and_return_conditional_losses_29253 inputs identity strided_slice/stackConst* _output_shapes :* dtype0*! valueB" 2 strided_slice/stackƒ strided_slice/stack_1Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_1ƒ strided_slice/stack_2Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_2ƒ strided_slice StridedSliceinputsstrided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* begin_mask* end_mask2 strided_slice SqueezeSqueezestrided_slice:output:0* T0*( _output_shapes : ’’’’’’’’’€* squeeze_dims 2 Squeezee IdentityIdentitySqueeze:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs ƒ ļ 3__inference_transformer/layer_4_layer_call_fn_27020 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_94562 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : †š   M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_8764 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_2/ExpandDims/dimõ *self_attention/masked_softmax_2/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_2/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_2/ExpandDims“ %self_attention/masked_softmax_2/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_2/sub/x‚ #self_attention/masked_softmax_2/subSub.self_attention/masked_softmax_2/sub/x:output:03self_attention/masked_softmax_2/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/sub“ %self_attention/masked_softmax_2/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_2/mul/yö #self_attention/masked_softmax_2/mulMul'self_attention/masked_softmax_2/sub:z:0.self_attention/masked_softmax_2/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/mulą #self_attention/masked_softmax_2/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/addŅ 'self_attention/masked_softmax_2/SoftmaxSoftmax'self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_2/SoftmaxŃ !self_attention/dropout_6/IdentityIdentity1self_attention/masked_softmax_2/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/dropout_6/Identity’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_6/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/add dropout_7/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/Identityx addAddV2inputsdropout_7/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_2/Pow/yž activation_2/PowPowintermediate/add:z:0activation_2/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Powm activation_2/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_2/mul/xž activation_2/mulMulactivation_2/mul/x:output:0activation_2/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul™ activation_2/addAddV2intermediate/add:z:0activation_2/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/addq activation_2/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_2/mul_1/x¤ activation_2/mul_1Mulactivation_2/mul_1/x:output:0activation_2/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_1† activation_2/TanhTanhactivation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Tanhq activation_2/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_2/add_1/x§ activation_2/add_1AddV2activation_2/add_1/x:output:0activation_2/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/add_1q activation_2/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_2/mul_2/x¦ activation_2/mul_2Mulactivation_2/mul_2/x:output:0activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_2 activation_2/mul_3Mulintermediate/add:z:0activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_2/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add„ dropout_8/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/Identity£ add_1AddV2dropout_8/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : † š 4__inference_transformer/layer_11_layer_call_fn_29235 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŅ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_118782 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ’ø   M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_7965 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¦ ,self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2. ,self_attention/masked_softmax/ExpandDims/dimļ (self_attention/masked_softmax/ExpandDims ExpandDimsinputs_15self_attention/masked_softmax/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax/ExpandDims #self_attention/masked_softmax/sub/xConst* _output_shapes :* dtype0* value B *€?2% #self_attention/masked_softmax/sub/xś !self_attention/masked_softmax/subSub,self_attention/masked_softmax/sub/x:output:01self_attention/masked_softmax/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/sub #self_attention/masked_softmax/mul/yConst* _output_shapes :* dtype0* value B *@Ę2% #self_attention/masked_softmax/mul/yī !self_attention/masked_softmax/mulMul%self_attention/masked_softmax/sub:z:0,self_attention/masked_softmax/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/mulŚ !self_attention/masked_softmax/addAddV2self_attention/Mul:z:0%self_attention/masked_softmax/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/addĢ %self_attention/masked_softmax/SoftmaxSoftmax%self_attention/masked_softmax/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/masked_softmax/Softmax‘ $self_attention/dropout/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2& $self_attention/dropout/dropout/Constū "self_attention/dropout/dropout/MulMul/self_attention/masked_softmax/Softmax:softmax:0-self_attention/dropout/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout/dropout/Mul« $self_attention/dropout/dropout/ShapeShape/self_attention/masked_softmax/Softmax:softmax:0* T0* _output_shapes :2& $self_attention/dropout/dropout/Shape“ ;self_attention/dropout/dropout/random_uniform/RandomUniform RandomUniform-self_attention/dropout/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02= ;self_attention/dropout/dropout/random_uniform/RandomUniform£ -self_attention/dropout/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2/ -self_attention/dropout/dropout/GreaterEqual/y“ +self_attention/dropout/dropout/GreaterEqual GreaterEqualDself_attention/dropout/dropout/random_uniform/RandomUniform:output:06self_attention/dropout/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2- +self_attention/dropout/dropout/GreaterEqualŽ #self_attention/dropout/dropout/CastCast/self_attention/dropout/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/dropout/dropout/Castš $self_attention/dropout/dropout/Mul_1Mul&self_attention/dropout/dropout/Mul:z:0'self_attention/dropout/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout/dropout/Mul_1ż self_attention/einsum_1/EinsumEinsum(self_attention/dropout/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addw dropout_1/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_1/dropout/ConstĄ dropout_1/dropout/MulMul'self_attention/attention_output/add:z:0 dropout_1/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/Mul‰ dropout_1/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_1/dropout/Shapeą .dropout_1/dropout/random_uniform/RandomUniform RandomUniform dropout_1/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_1/dropout/random_uniform/RandomUniform‰ dropout_1/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_1/dropout/GreaterEqual/yō dropout_1/dropout/GreaterEqual GreaterEqual7dropout_1/dropout/random_uniform/RandomUniform:output:0)dropout_1/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/GreaterEqual« dropout_1/dropout/CastCast"dropout_1/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/Cast° dropout_1/dropout/Mul_1Muldropout_1/dropout/Mul:z:0dropout_1/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/dropout/Mul_1x addAddV2inputsdropout_1/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addi activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation/Pow/y˜ activation/PowPowintermediate/add:z:0activation/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Powi activation/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation/mul/x– activation/mulMulactivation/mul/x:output:0activation/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul“ activation/addAddV2intermediate/add:z:0activation/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/addm activation/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation/mul_1/xœ activation/mul_1Mulactivation/mul_1/x:output:0activation/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_1€ activation/TanhTanhactivation/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Tanhm activation/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation/add_1/xŸ activation/add_1AddV2activation/add_1/x:output:0activation/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/add_1m activation/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation/mul_2/xž activation/mul_2Mulactivation/mul_2/x:output:0activation/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_2— activation/mul_3Mulintermediate/add:z:0activation/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpŻ output/einsum/EinsumEinsumactivation/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addw dropout_2/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_2/dropout/Const§ dropout_2/dropout/MulMuloutput/add:z:0 dropout_2/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/Mulp dropout_2/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_2/dropout/Shapeą .dropout_2/dropout/random_uniform/RandomUniform RandomUniform dropout_2/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_2/dropout/random_uniform/RandomUniform‰ dropout_2/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_2/dropout/GreaterEqual/yō dropout_2/dropout/GreaterEqual GreaterEqual7dropout_2/dropout/random_uniform/RandomUniform:output:0)dropout_2/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/GreaterEqual« dropout_2/dropout/CastCast"dropout_2/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/Cast° dropout_2/dropout/Mul_1Muldropout_2/dropout/Mul:z:0dropout_2/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/dropout/Mul_1£ add_1AddV2dropout_2/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ö ] #__inference_add_layer_call_fn_25402 inputs_0 inputs_1 inputs_2 identityŽ PartitionedCallPartitionedCallinputs_0inputs_1inputs_2* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*F fAR? =__inference_add_layer_call_and_return_conditional_losses_77132 PartitionedCallz IdentityIdentityPartitionedCall:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*v _input_shapese c: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/1:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/2 ‹q ę ,__inference_functional_1_layer_call_fn_16586 input_word_ids input_mask input_type_ids unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall® StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idsunknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*P fKRI G__inference_functional_1_layer_call_and_return_conditional_losses_161832 StatefulPartitionedCall IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity  Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : Ž ] A__inference_lambda_layer_call_and_return_conditional_losses_12007 inputs identity strided_slice/stackConst* _output_shapes :* dtype0*! valueB" 2 strided_slice/stackƒ strided_slice/stack_1Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_1ƒ strided_slice/stack_2Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_2ƒ strided_slice StridedSliceinputsstrided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* begin_mask* end_mask2 strided_slice SqueezeSqueezestrided_slice:output:0* T0*( _output_shapes : ’’’’’’’’’€* squeeze_dims 2 Squeezee IdentityIdentitySqueeze:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs ™q ķ 3__inference_transformer_encoder_layer_call_fn_13309 input_word_ids input_mask input_type_ids unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCallµ StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idsunknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_129062 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity“ Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : žš £ N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_28226 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_8/ExpandDims/dimõ *self_attention/masked_softmax_8/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_8/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_8/ExpandDims“ %self_attention/masked_softmax_8/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_8/sub/x‚ #self_attention/masked_softmax_8/subSub.self_attention/masked_softmax_8/sub/x:output:03self_attention/masked_softmax_8/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/sub“ %self_attention/masked_softmax_8/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_8/mul/yö #self_attention/masked_softmax_8/mulMul'self_attention/masked_softmax_8/sub:z:0.self_attention/masked_softmax_8/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/mulą #self_attention/masked_softmax_8/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/addŅ 'self_attention/masked_softmax_8/SoftmaxSoftmax'self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_8/SoftmaxÓ "self_attention/dropout_24/IdentityIdentity1self_attention/masked_softmax_8/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_24/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_24/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_25/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/Identity{ addAddV2inputs_0dropout_25/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_8/Pow/yž activation_8/PowPowintermediate/add:z:0activation_8/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Powm activation_8/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_8/mul/xž activation_8/mulMulactivation_8/mul/x:output:0activation_8/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul™ activation_8/addAddV2intermediate/add:z:0activation_8/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/addq activation_8/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_8/mul_1/x¤ activation_8/mul_1Mulactivation_8/mul_1/x:output:0activation_8/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_1† activation_8/TanhTanhactivation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Tanhq activation_8/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_8/add_1/x§ activation_8/add_1AddV2activation_8/add_1/x:output:0activation_8/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/add_1q activation_8/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_8/mul_2/x¦ activation_8/mul_2Mulactivation_8/mul_2/x:output:0activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_2 activation_8/mul_3Mulintermediate/add:z:0activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_8/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_26/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/Identity¤ add_1AddV2dropout_26/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : … Ķ7 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_13739 inputs inputs_1 inputs_2 word_embeddings_13318 position_embedding_13321 type_embeddings_13324 embeddings_layer_norm_13328 embeddings_layer_norm_13330 transformer_layer_0_13335 transformer_layer_0_13337 transformer_layer_0_13339 transformer_layer_0_13341 transformer_layer_0_13343 transformer_layer_0_13345 transformer_layer_0_13347 transformer_layer_0_13349 transformer_layer_0_13351 transformer_layer_0_13353 transformer_layer_0_13355 transformer_layer_0_13357 transformer_layer_0_13359 transformer_layer_0_13361 transformer_layer_0_13363 transformer_layer_0_13365 transformer_layer_1_13368 transformer_layer_1_13370 transformer_layer_1_13372 transformer_layer_1_13374 transformer_layer_1_13376 transformer_layer_1_13378 transformer_layer_1_13380 transformer_layer_1_13382 transformer_layer_1_13384 transformer_layer_1_13386 transformer_layer_1_13388 transformer_layer_1_13390 transformer_layer_1_13392 transformer_layer_1_13394 transformer_layer_1_13396 transformer_layer_1_13398 transformer_layer_2_13401 transformer_layer_2_13403 transformer_layer_2_13405 transformer_layer_2_13407 transformer_layer_2_13409 transformer_layer_2_13411 transformer_layer_2_13413 transformer_layer_2_13415 transformer_layer_2_13417 transformer_layer_2_13419 transformer_layer_2_13421 transformer_layer_2_13423 transformer_layer_2_13425 transformer_layer_2_13427 transformer_layer_2_13429 transformer_layer_2_13431 transformer_layer_3_13434 transformer_layer_3_13436 transformer_layer_3_13438 transformer_layer_3_13440 transformer_layer_3_13442 transformer_layer_3_13444 transformer_layer_3_13446 transformer_layer_3_13448 transformer_layer_3_13450 transformer_layer_3_13452 transformer_layer_3_13454 transformer_layer_3_13456 transformer_layer_3_13458 transformer_layer_3_13460 transformer_layer_3_13462 transformer_layer_3_13464 transformer_layer_4_13467 transformer_layer_4_13469 transformer_layer_4_13471 transformer_layer_4_13473 transformer_layer_4_13475 transformer_layer_4_13477 transformer_layer_4_13479 transformer_layer_4_13481 transformer_layer_4_13483 transformer_layer_4_13485 transformer_layer_4_13487 transformer_layer_4_13489 transformer_layer_4_13491 transformer_layer_4_13493 transformer_layer_4_13495 transformer_layer_4_13497 transformer_layer_5_13500 transformer_layer_5_13502 transformer_layer_5_13504 transformer_layer_5_13506 transformer_layer_5_13508 transformer_layer_5_13510 transformer_layer_5_13512 transformer_layer_5_13514 transformer_layer_5_13516 transformer_layer_5_13518 transformer_layer_5_13520 transformer_layer_5_13522 transformer_layer_5_13524 transformer_layer_5_13526 transformer_layer_5_13528 transformer_layer_5_13530 transformer_layer_6_13533 transformer_layer_6_13535 transformer_layer_6_13537 transformer_layer_6_13539 transformer_layer_6_13541 transformer_layer_6_13543 transformer_layer_6_13545 transformer_layer_6_13547 transformer_layer_6_13549 transformer_layer_6_13551 transformer_layer_6_13553 transformer_layer_6_13555 transformer_layer_6_13557 transformer_layer_6_13559 transformer_layer_6_13561 transformer_layer_6_13563 transformer_layer_7_13566 transformer_layer_7_13568 transformer_layer_7_13570 transformer_layer_7_13572 transformer_layer_7_13574 transformer_layer_7_13576 transformer_layer_7_13578 transformer_layer_7_13580 transformer_layer_7_13582 transformer_layer_7_13584 transformer_layer_7_13586 transformer_layer_7_13588 transformer_layer_7_13590 transformer_layer_7_13592 transformer_layer_7_13594 transformer_layer_7_13596 transformer_layer_8_13599 transformer_layer_8_13601 transformer_layer_8_13603 transformer_layer_8_13605 transformer_layer_8_13607 transformer_layer_8_13609 transformer_layer_8_13611 transformer_layer_8_13613 transformer_layer_8_13615 transformer_layer_8_13617 transformer_layer_8_13619 transformer_layer_8_13621 transformer_layer_8_13623 transformer_layer_8_13625 transformer_layer_8_13627 transformer_layer_8_13629 transformer_layer_9_13632 transformer_layer_9_13634 transformer_layer_9_13636 transformer_layer_9_13638 transformer_layer_9_13640 transformer_layer_9_13642 transformer_layer_9_13644 transformer_layer_9_13646 transformer_layer_9_13648 transformer_layer_9_13650 transformer_layer_9_13652 transformer_layer_9_13654 transformer_layer_9_13656 transformer_layer_9_13658 transformer_layer_9_13660 transformer_layer_9_13662 transformer_layer_10_13665 transformer_layer_10_13667 transformer_layer_10_13669 transformer_layer_10_13671 transformer_layer_10_13673 transformer_layer_10_13675 transformer_layer_10_13677 transformer_layer_10_13679 transformer_layer_10_13681 transformer_layer_10_13683 transformer_layer_10_13685 transformer_layer_10_13687 transformer_layer_10_13689 transformer_layer_10_13691 transformer_layer_10_13693 transformer_layer_10_13695 transformer_layer_11_13698 transformer_layer_11_13700 transformer_layer_11_13702 transformer_layer_11_13704 transformer_layer_11_13706 transformer_layer_11_13708 transformer_layer_11_13710 transformer_layer_11_13712 transformer_layer_11_13714 transformer_layer_11_13716 transformer_layer_11_13718 transformer_layer_11_13720 transformer_layer_11_13722 transformer_layer_11_13724 transformer_layer_11_13726 transformer_layer_11_13728 pooler_transform_13732 pooler_transform_13734 identity identity_1ˆ¢-embeddings/layer_norm/StatefulPartitionedCall¢(pooler_transform/StatefulPartitionedCall¢*position_embedding/StatefulPartitionedCall¢+transformer/layer_0/StatefulPartitionedCall¢+transformer/layer_1/StatefulPartitionedCall¢,transformer/layer_10/StatefulPartitionedCall¢,transformer/layer_11/StatefulPartitionedCall¢+transformer/layer_2/StatefulPartitionedCall¢+transformer/layer_3/StatefulPartitionedCall¢+transformer/layer_4/StatefulPartitionedCall¢+transformer/layer_5/StatefulPartitionedCall¢+transformer/layer_6/StatefulPartitionedCall¢+transformer/layer_7/StatefulPartitionedCall¢+transformer/layer_8/StatefulPartitionedCall¢+transformer/layer_9/StatefulPartitionedCall¢'type_embeddings/StatefulPartitionedCall¢'word_embeddings/StatefulPartitionedCall„ 'word_embeddings/StatefulPartitionedCallStatefulPartitionedCallinputsword_embeddings_13318* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_word_embeddings_layer_call_and_return_conditional_losses_76282) 'word_embeddings/StatefulPartitionedCallŪ *position_embedding/StatefulPartitionedCallStatefulPartitionedCall0word_embeddings/StatefulPartitionedCall:output:0position_embedding_13321* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*U fPRN L__inference_position_embedding_layer_call_and_return_conditional_losses_76632, *position_embedding/StatefulPartitionedCall§ 'type_embeddings/StatefulPartitionedCallStatefulPartitionedCallinputs_2type_embeddings_13324* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_type_embeddings_layer_call_and_return_conditional_losses_76932) 'type_embeddings/StatefulPartitionedCallį add/PartitionedCallPartitionedCall0word_embeddings/StatefulPartitionedCall:output:03position_embedding/StatefulPartitionedCall:output:00type_embeddings/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*F fAR? =__inference_add_layer_call_and_return_conditional_losses_77132 add/PartitionedCallņ -embeddings/layer_norm/StatefulPartitionedCallStatefulPartitionedCalladd/PartitionedCall:output:0embeddings_layer_norm_13328embeddings_layer_norm_13330* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*X fSRQ O__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_77452/ -embeddings/layer_norm/StatefulPartitionedCallŠ dropout/PartitionedCallPartitionedCall6embeddings/layer_norm/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_dropout_layer_call_and_return_conditional_losses_77782 dropout/PartitionedCall« #self_attention_mask/PartitionedCallPartitionedCall dropout/PartitionedCall:output:0inputs_1* Tin 2* Tout 2* _collective_manager_ids *= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*V fQRO M__inference_self_attention_mask_layer_call_and_return_conditional_losses_78282% #self_attention_mask/PartitionedCall± +transformer/layer_0/StatefulPartitionedCallStatefulPartitionedCall dropout/PartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_0_13335transformer_layer_0_13337transformer_layer_0_13339transformer_layer_0_13341transformer_layer_0_13343transformer_layer_0_13345transformer_layer_0_13347transformer_layer_0_13349transformer_layer_0_13351transformer_layer_0_13353transformer_layer_0_13355transformer_layer_0_13357transformer_layer_0_13359transformer_layer_0_13361transformer_layer_0_13363transformer_layer_0_13365* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_80722- +transformer/layer_0/StatefulPartitionedCallÅ +transformer/layer_1/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_0/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_1_13368transformer_layer_1_13370transformer_layer_1_13372transformer_layer_1_13374transformer_layer_1_13376transformer_layer_1_13378transformer_layer_1_13380transformer_layer_1_13382transformer_layer_1_13384transformer_layer_1_13386transformer_layer_1_13388transformer_layer_1_13390transformer_layer_1_13392transformer_layer_1_13394transformer_layer_1_13396transformer_layer_1_13398* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_84182- +transformer/layer_1/StatefulPartitionedCallÅ +transformer/layer_2/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_1/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_2_13401transformer_layer_2_13403transformer_layer_2_13405transformer_layer_2_13407transformer_layer_2_13409transformer_layer_2_13411transformer_layer_2_13413transformer_layer_2_13415transformer_layer_2_13417transformer_layer_2_13419transformer_layer_2_13421transformer_layer_2_13423transformer_layer_2_13425transformer_layer_2_13427transformer_layer_2_13429transformer_layer_2_13431* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_87642- +transformer/layer_2/StatefulPartitionedCallÅ +transformer/layer_3/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_2/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_3_13434transformer_layer_3_13436transformer_layer_3_13438transformer_layer_3_13440transformer_layer_3_13442transformer_layer_3_13444transformer_layer_3_13446transformer_layer_3_13448transformer_layer_3_13450transformer_layer_3_13452transformer_layer_3_13454transformer_layer_3_13456transformer_layer_3_13458transformer_layer_3_13460transformer_layer_3_13462transformer_layer_3_13464* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_91102- +transformer/layer_3/StatefulPartitionedCallÅ +transformer/layer_4/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_3/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_4_13467transformer_layer_4_13469transformer_layer_4_13471transformer_layer_4_13473transformer_layer_4_13475transformer_layer_4_13477transformer_layer_4_13479transformer_layer_4_13481transformer_layer_4_13483transformer_layer_4_13485transformer_layer_4_13487transformer_layer_4_13489transformer_layer_4_13491transformer_layer_4_13493transformer_layer_4_13495transformer_layer_4_13497* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_94562- +transformer/layer_4/StatefulPartitionedCallÅ +transformer/layer_5/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_4/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_5_13500transformer_layer_5_13502transformer_layer_5_13504transformer_layer_5_13506transformer_layer_5_13508transformer_layer_5_13510transformer_layer_5_13512transformer_layer_5_13514transformer_layer_5_13516transformer_layer_5_13518transformer_layer_5_13520transformer_layer_5_13522transformer_layer_5_13524transformer_layer_5_13526transformer_layer_5_13528transformer_layer_5_13530* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_98022- +transformer/layer_5/StatefulPartitionedCallĘ +transformer/layer_6/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_5/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_6_13533transformer_layer_6_13535transformer_layer_6_13537transformer_layer_6_13539transformer_layer_6_13541transformer_layer_6_13543transformer_layer_6_13545transformer_layer_6_13547transformer_layer_6_13549transformer_layer_6_13551transformer_layer_6_13553transformer_layer_6_13555transformer_layer_6_13557transformer_layer_6_13559transformer_layer_6_13561transformer_layer_6_13563* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_101482- +transformer/layer_6/StatefulPartitionedCallĘ +transformer/layer_7/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_6/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_7_13566transformer_layer_7_13568transformer_layer_7_13570transformer_layer_7_13572transformer_layer_7_13574transformer_layer_7_13576transformer_layer_7_13578transformer_layer_7_13580transformer_layer_7_13582transformer_layer_7_13584transformer_layer_7_13586transformer_layer_7_13588transformer_layer_7_13590transformer_layer_7_13592transformer_layer_7_13594transformer_layer_7_13596* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_104942- +transformer/layer_7/StatefulPartitionedCallĘ +transformer/layer_8/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_7/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_8_13599transformer_layer_8_13601transformer_layer_8_13603transformer_layer_8_13605transformer_layer_8_13607transformer_layer_8_13609transformer_layer_8_13611transformer_layer_8_13613transformer_layer_8_13615transformer_layer_8_13617transformer_layer_8_13619transformer_layer_8_13621transformer_layer_8_13623transformer_layer_8_13625transformer_layer_8_13627transformer_layer_8_13629* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_108402- +transformer/layer_8/StatefulPartitionedCallĘ +transformer/layer_9/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_8/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_9_13632transformer_layer_9_13634transformer_layer_9_13636transformer_layer_9_13638transformer_layer_9_13640transformer_layer_9_13642transformer_layer_9_13644transformer_layer_9_13646transformer_layer_9_13648transformer_layer_9_13650transformer_layer_9_13652transformer_layer_9_13654transformer_layer_9_13656transformer_layer_9_13658transformer_layer_9_13660transformer_layer_9_13662* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_111862- +transformer/layer_9/StatefulPartitionedCallŁ ,transformer/layer_10/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_9/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_10_13665transformer_layer_10_13667transformer_layer_10_13669transformer_layer_10_13671transformer_layer_10_13673transformer_layer_10_13675transformer_layer_10_13677transformer_layer_10_13679transformer_layer_10_13681transformer_layer_10_13683transformer_layer_10_13685transformer_layer_10_13687transformer_layer_10_13689transformer_layer_10_13691transformer_layer_10_13693transformer_layer_10_13695* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_115322. ,transformer/layer_10/StatefulPartitionedCallŚ ,transformer/layer_11/StatefulPartitionedCallStatefulPartitionedCall5transformer/layer_10/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_11_13698transformer_layer_11_13700transformer_layer_11_13702transformer_layer_11_13704transformer_layer_11_13706transformer_layer_11_13708transformer_layer_11_13710transformer_layer_11_13712transformer_layer_11_13714transformer_layer_11_13716transformer_layer_11_13718transformer_layer_11_13720transformer_layer_11_13722transformer_layer_11_13724transformer_layer_11_13726transformer_layer_11_13728* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_118782. ,transformer/layer_11/StatefulPartitionedCallś lambda/PartitionedCallPartitionedCall5transformer/layer_11/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_lambda_layer_call_and_return_conditional_losses_120072 lambda/PartitionedCallŠ (pooler_transform/StatefulPartitionedCallStatefulPartitionedCalllambda/PartitionedCall:output:0pooler_transform_13732pooler_transform_13734* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*T fORM K__inference_pooler_transform_layer_call_and_return_conditional_losses_120312* (pooler_transform/StatefulPartitionedCall IdentityIdentity5transformer/layer_11/StatefulPartitionedCall:output:0.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity Identity_1Identity1pooler_transform/StatefulPartitionedCall:output:0.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2^ -embeddings/layer_norm/StatefulPartitionedCall-embeddings/layer_norm/StatefulPartitionedCall2T (pooler_transform/StatefulPartitionedCall(pooler_transform/StatefulPartitionedCall2X *position_embedding/StatefulPartitionedCall*position_embedding/StatefulPartitionedCall2Z +transformer/layer_0/StatefulPartitionedCall+transformer/layer_0/StatefulPartitionedCall2Z +transformer/layer_1/StatefulPartitionedCall+transformer/layer_1/StatefulPartitionedCall2\ ,transformer/layer_10/StatefulPartitionedCall,transformer/layer_10/StatefulPartitionedCall2\ ,transformer/layer_11/StatefulPartitionedCall,transformer/layer_11/StatefulPartitionedCall2Z +transformer/layer_2/StatefulPartitionedCall+transformer/layer_2/StatefulPartitionedCall2Z +transformer/layer_3/StatefulPartitionedCall+transformer/layer_3/StatefulPartitionedCall2Z +transformer/layer_4/StatefulPartitionedCall+transformer/layer_4/StatefulPartitionedCall2Z +transformer/layer_5/StatefulPartitionedCall+transformer/layer_5/StatefulPartitionedCall2Z +transformer/layer_6/StatefulPartitionedCall+transformer/layer_6/StatefulPartitionedCall2Z +transformer/layer_7/StatefulPartitionedCall+transformer/layer_7/StatefulPartitionedCall2Z +transformer/layer_8/StatefulPartitionedCall+transformer/layer_8/StatefulPartitionedCall2Z +transformer/layer_9/StatefulPartitionedCall+transformer/layer_9/StatefulPartitionedCall2R 'type_embeddings/StatefulPartitionedCall'type_embeddings/StatefulPartitionedCall2R 'word_embeddings/StatefulPartitionedCall'word_embeddings/StatefulPartitionedCall:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : „ ļ 3__inference_transformer/layer_8_layer_call_fn_28264 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_108402 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ģ… š7 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12049 input_word_ids input_mask input_type_ids word_embeddings_7637 position_embedding_7672 type_embeddings_7702 embeddings_layer_norm_7756 embeddings_layer_norm_7758 transformer_layer_0_8150 transformer_layer_0_8152 transformer_layer_0_8154 transformer_layer_0_8156 transformer_layer_0_8158 transformer_layer_0_8160 transformer_layer_0_8162 transformer_layer_0_8164 transformer_layer_0_8166 transformer_layer_0_8168 transformer_layer_0_8170 transformer_layer_0_8172 transformer_layer_0_8174 transformer_layer_0_8176 transformer_layer_0_8178 transformer_layer_0_8180 transformer_layer_1_8496 transformer_layer_1_8498 transformer_layer_1_8500 transformer_layer_1_8502 transformer_layer_1_8504 transformer_layer_1_8506 transformer_layer_1_8508 transformer_layer_1_8510 transformer_layer_1_8512 transformer_layer_1_8514 transformer_layer_1_8516 transformer_layer_1_8518 transformer_layer_1_8520 transformer_layer_1_8522 transformer_layer_1_8524 transformer_layer_1_8526 transformer_layer_2_8842 transformer_layer_2_8844 transformer_layer_2_8846 transformer_layer_2_8848 transformer_layer_2_8850 transformer_layer_2_8852 transformer_layer_2_8854 transformer_layer_2_8856 transformer_layer_2_8858 transformer_layer_2_8860 transformer_layer_2_8862 transformer_layer_2_8864 transformer_layer_2_8866 transformer_layer_2_8868 transformer_layer_2_8870 transformer_layer_2_8872 transformer_layer_3_9188 transformer_layer_3_9190 transformer_layer_3_9192 transformer_layer_3_9194 transformer_layer_3_9196 transformer_layer_3_9198 transformer_layer_3_9200 transformer_layer_3_9202 transformer_layer_3_9204 transformer_layer_3_9206 transformer_layer_3_9208 transformer_layer_3_9210 transformer_layer_3_9212 transformer_layer_3_9214 transformer_layer_3_9216 transformer_layer_3_9218 transformer_layer_4_9534 transformer_layer_4_9536 transformer_layer_4_9538 transformer_layer_4_9540 transformer_layer_4_9542 transformer_layer_4_9544 transformer_layer_4_9546 transformer_layer_4_9548 transformer_layer_4_9550 transformer_layer_4_9552 transformer_layer_4_9554 transformer_layer_4_9556 transformer_layer_4_9558 transformer_layer_4_9560 transformer_layer_4_9562 transformer_layer_4_9564 transformer_layer_5_9880 transformer_layer_5_9882 transformer_layer_5_9884 transformer_layer_5_9886 transformer_layer_5_9888 transformer_layer_5_9890 transformer_layer_5_9892 transformer_layer_5_9894 transformer_layer_5_9896 transformer_layer_5_9898 transformer_layer_5_9900 transformer_layer_5_9902 transformer_layer_5_9904 transformer_layer_5_9906 transformer_layer_5_9908 transformer_layer_5_9910 transformer_layer_6_10226 transformer_layer_6_10228 transformer_layer_6_10230 transformer_layer_6_10232 transformer_layer_6_10234 transformer_layer_6_10236 transformer_layer_6_10238 transformer_layer_6_10240 transformer_layer_6_10242 transformer_layer_6_10244 transformer_layer_6_10246 transformer_layer_6_10248 transformer_layer_6_10250 transformer_layer_6_10252 transformer_layer_6_10254 transformer_layer_6_10256 transformer_layer_7_10572 transformer_layer_7_10574 transformer_layer_7_10576 transformer_layer_7_10578 transformer_layer_7_10580 transformer_layer_7_10582 transformer_layer_7_10584 transformer_layer_7_10586 transformer_layer_7_10588 transformer_layer_7_10590 transformer_layer_7_10592 transformer_layer_7_10594 transformer_layer_7_10596 transformer_layer_7_10598 transformer_layer_7_10600 transformer_layer_7_10602 transformer_layer_8_10918 transformer_layer_8_10920 transformer_layer_8_10922 transformer_layer_8_10924 transformer_layer_8_10926 transformer_layer_8_10928 transformer_layer_8_10930 transformer_layer_8_10932 transformer_layer_8_10934 transformer_layer_8_10936 transformer_layer_8_10938 transformer_layer_8_10940 transformer_layer_8_10942 transformer_layer_8_10944 transformer_layer_8_10946 transformer_layer_8_10948 transformer_layer_9_11264 transformer_layer_9_11266 transformer_layer_9_11268 transformer_layer_9_11270 transformer_layer_9_11272 transformer_layer_9_11274 transformer_layer_9_11276 transformer_layer_9_11278 transformer_layer_9_11280 transformer_layer_9_11282 transformer_layer_9_11284 transformer_layer_9_11286 transformer_layer_9_11288 transformer_layer_9_11290 transformer_layer_9_11292 transformer_layer_9_11294 transformer_layer_10_11610 transformer_layer_10_11612 transformer_layer_10_11614 transformer_layer_10_11616 transformer_layer_10_11618 transformer_layer_10_11620 transformer_layer_10_11622 transformer_layer_10_11624 transformer_layer_10_11626 transformer_layer_10_11628 transformer_layer_10_11630 transformer_layer_10_11632 transformer_layer_10_11634 transformer_layer_10_11636 transformer_layer_10_11638 transformer_layer_10_11640 transformer_layer_11_11956 transformer_layer_11_11958 transformer_layer_11_11960 transformer_layer_11_11962 transformer_layer_11_11964 transformer_layer_11_11966 transformer_layer_11_11968 transformer_layer_11_11970 transformer_layer_11_11972 transformer_layer_11_11974 transformer_layer_11_11976 transformer_layer_11_11978 transformer_layer_11_11980 transformer_layer_11_11982 transformer_layer_11_11984 transformer_layer_11_11986 pooler_transform_12042 pooler_transform_12044 identity identity_1ˆ¢dropout/StatefulPartitionedCall¢-embeddings/layer_norm/StatefulPartitionedCall¢(pooler_transform/StatefulPartitionedCall¢*position_embedding/StatefulPartitionedCall¢+transformer/layer_0/StatefulPartitionedCall¢+transformer/layer_1/StatefulPartitionedCall¢,transformer/layer_10/StatefulPartitionedCall¢,transformer/layer_11/StatefulPartitionedCall¢+transformer/layer_2/StatefulPartitionedCall¢+transformer/layer_3/StatefulPartitionedCall¢+transformer/layer_4/StatefulPartitionedCall¢+transformer/layer_5/StatefulPartitionedCall¢+transformer/layer_6/StatefulPartitionedCall¢+transformer/layer_7/StatefulPartitionedCall¢+transformer/layer_8/StatefulPartitionedCall¢+transformer/layer_9/StatefulPartitionedCall¢'type_embeddings/StatefulPartitionedCall¢'word_embeddings/StatefulPartitionedCall¬ 'word_embeddings/StatefulPartitionedCallStatefulPartitionedCallinput_word_idsword_embeddings_7637* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_word_embeddings_layer_call_and_return_conditional_losses_76282) 'word_embeddings/StatefulPartitionedCallŚ *position_embedding/StatefulPartitionedCallStatefulPartitionedCall0word_embeddings/StatefulPartitionedCall:output:0position_embedding_7672* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*U fPRN L__inference_position_embedding_layer_call_and_return_conditional_losses_76632, *position_embedding/StatefulPartitionedCall¬ 'type_embeddings/StatefulPartitionedCallStatefulPartitionedCallinput_type_idstype_embeddings_7702* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_type_embeddings_layer_call_and_return_conditional_losses_76932) 'type_embeddings/StatefulPartitionedCallį add/PartitionedCallPartitionedCall0word_embeddings/StatefulPartitionedCall:output:03position_embedding/StatefulPartitionedCall:output:00type_embeddings/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*F fAR? =__inference_add_layer_call_and_return_conditional_losses_77132 add/PartitionedCallš -embeddings/layer_norm/StatefulPartitionedCallStatefulPartitionedCalladd/PartitionedCall:output:0embeddings_layer_norm_7756embeddings_layer_norm_7758* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*X fSRQ O__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_77452/ -embeddings/layer_norm/StatefulPartitionedCall¢ dropout/StatefulPartitionedCallStatefulPartitionedCall6embeddings/layer_norm/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_dropout_layer_call_and_return_conditional_losses_77732! dropout/StatefulPartitionedCallµ #self_attention_mask/PartitionedCallPartitionedCall(dropout/StatefulPartitionedCall:output:0 input_mask* Tin 2* Tout 2* _collective_manager_ids *= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*V fQRO M__inference_self_attention_mask_layer_call_and_return_conditional_losses_78282% #self_attention_mask/PartitionedCall© +transformer/layer_0/StatefulPartitionedCallStatefulPartitionedCall(dropout/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_0_8150transformer_layer_0_8152transformer_layer_0_8154transformer_layer_0_8156transformer_layer_0_8158transformer_layer_0_8160transformer_layer_0_8162transformer_layer_0_8164transformer_layer_0_8166transformer_layer_0_8168transformer_layer_0_8170transformer_layer_0_8172transformer_layer_0_8174transformer_layer_0_8176transformer_layer_0_8178transformer_layer_0_8180* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_0_layer_call_and_return_conditional_losses_79652- +transformer/layer_0/StatefulPartitionedCallµ +transformer/layer_1/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_0/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_1_8496transformer_layer_1_8498transformer_layer_1_8500transformer_layer_1_8502transformer_layer_1_8504transformer_layer_1_8506transformer_layer_1_8508transformer_layer_1_8510transformer_layer_1_8512transformer_layer_1_8514transformer_layer_1_8516transformer_layer_1_8518transformer_layer_1_8520transformer_layer_1_8522transformer_layer_1_8524transformer_layer_1_8526* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_83112- +transformer/layer_1/StatefulPartitionedCallµ +transformer/layer_2/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_1/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_2_8842transformer_layer_2_8844transformer_layer_2_8846transformer_layer_2_8848transformer_layer_2_8850transformer_layer_2_8852transformer_layer_2_8854transformer_layer_2_8856transformer_layer_2_8858transformer_layer_2_8860transformer_layer_2_8862transformer_layer_2_8864transformer_layer_2_8866transformer_layer_2_8868transformer_layer_2_8870transformer_layer_2_8872* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_86572- +transformer/layer_2/StatefulPartitionedCallµ +transformer/layer_3/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_2/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_3_9188transformer_layer_3_9190transformer_layer_3_9192transformer_layer_3_9194transformer_layer_3_9196transformer_layer_3_9198transformer_layer_3_9200transformer_layer_3_9202transformer_layer_3_9204transformer_layer_3_9206transformer_layer_3_9208transformer_layer_3_9210transformer_layer_3_9212transformer_layer_3_9214transformer_layer_3_9216transformer_layer_3_9218* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_90032- +transformer/layer_3/StatefulPartitionedCallµ +transformer/layer_4/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_3/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_4_9534transformer_layer_4_9536transformer_layer_4_9538transformer_layer_4_9540transformer_layer_4_9542transformer_layer_4_9544transformer_layer_4_9546transformer_layer_4_9548transformer_layer_4_9550transformer_layer_4_9552transformer_layer_4_9554transformer_layer_4_9556transformer_layer_4_9558transformer_layer_4_9560transformer_layer_4_9562transformer_layer_4_9564* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_4_layer_call_and_return_conditional_losses_93492- +transformer/layer_4/StatefulPartitionedCallµ +transformer/layer_5/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_4/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_5_9880transformer_layer_5_9882transformer_layer_5_9884transformer_layer_5_9886transformer_layer_5_9888transformer_layer_5_9890transformer_layer_5_9892transformer_layer_5_9894transformer_layer_5_9896transformer_layer_5_9898transformer_layer_5_9900transformer_layer_5_9902transformer_layer_5_9904transformer_layer_5_9906transformer_layer_5_9908transformer_layer_5_9910* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_96952- +transformer/layer_5/StatefulPartitionedCallĘ +transformer/layer_6/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_5/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_6_10226transformer_layer_6_10228transformer_layer_6_10230transformer_layer_6_10232transformer_layer_6_10234transformer_layer_6_10236transformer_layer_6_10238transformer_layer_6_10240transformer_layer_6_10242transformer_layer_6_10244transformer_layer_6_10246transformer_layer_6_10248transformer_layer_6_10250transformer_layer_6_10252transformer_layer_6_10254transformer_layer_6_10256* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_100412- +transformer/layer_6/StatefulPartitionedCallĘ +transformer/layer_7/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_6/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_7_10572transformer_layer_7_10574transformer_layer_7_10576transformer_layer_7_10578transformer_layer_7_10580transformer_layer_7_10582transformer_layer_7_10584transformer_layer_7_10586transformer_layer_7_10588transformer_layer_7_10590transformer_layer_7_10592transformer_layer_7_10594transformer_layer_7_10596transformer_layer_7_10598transformer_layer_7_10600transformer_layer_7_10602* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_103872- +transformer/layer_7/StatefulPartitionedCallĘ +transformer/layer_8/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_7/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_8_10918transformer_layer_8_10920transformer_layer_8_10922transformer_layer_8_10924transformer_layer_8_10926transformer_layer_8_10928transformer_layer_8_10930transformer_layer_8_10932transformer_layer_8_10934transformer_layer_8_10936transformer_layer_8_10938transformer_layer_8_10940transformer_layer_8_10942transformer_layer_8_10944transformer_layer_8_10946transformer_layer_8_10948* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_107332- +transformer/layer_8/StatefulPartitionedCallĘ +transformer/layer_9/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_8/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_9_11264transformer_layer_9_11266transformer_layer_9_11268transformer_layer_9_11270transformer_layer_9_11272transformer_layer_9_11274transformer_layer_9_11276transformer_layer_9_11278transformer_layer_9_11280transformer_layer_9_11282transformer_layer_9_11284transformer_layer_9_11286transformer_layer_9_11288transformer_layer_9_11290transformer_layer_9_11292transformer_layer_9_11294* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_110792- +transformer/layer_9/StatefulPartitionedCallŁ ,transformer/layer_10/StatefulPartitionedCallStatefulPartitionedCall4transformer/layer_9/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_10_11610transformer_layer_10_11612transformer_layer_10_11614transformer_layer_10_11616transformer_layer_10_11618transformer_layer_10_11620transformer_layer_10_11622transformer_layer_10_11624transformer_layer_10_11626transformer_layer_10_11628transformer_layer_10_11630transformer_layer_10_11632transformer_layer_10_11634transformer_layer_10_11636transformer_layer_10_11638transformer_layer_10_11640* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_114252. ,transformer/layer_10/StatefulPartitionedCallŚ ,transformer/layer_11/StatefulPartitionedCallStatefulPartitionedCall5transformer/layer_10/StatefulPartitionedCall:output:0,self_attention_mask/PartitionedCall:output:0transformer_layer_11_11956transformer_layer_11_11958transformer_layer_11_11960transformer_layer_11_11962transformer_layer_11_11964transformer_layer_11_11966transformer_layer_11_11968transformer_layer_11_11970transformer_layer_11_11972transformer_layer_11_11974transformer_layer_11_11976transformer_layer_11_11978transformer_layer_11_11980transformer_layer_11_11982transformer_layer_11_11984transformer_layer_11_11986* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_117712. ,transformer/layer_11/StatefulPartitionedCallś lambda/PartitionedCallPartitionedCall5transformer/layer_11/StatefulPartitionedCall:output:0* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_lambda_layer_call_and_return_conditional_losses_119982 lambda/PartitionedCallŠ (pooler_transform/StatefulPartitionedCallStatefulPartitionedCalllambda/PartitionedCall:output:0pooler_transform_12042pooler_transform_12044* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€*$ _read_only_resource_inputs ** config_proto  CPU  GPU2J8*T fORM K__inference_pooler_transform_layer_call_and_return_conditional_losses_120312* (pooler_transform/StatefulPartitionedCallæ IdentityIdentity5transformer/layer_11/StatefulPartitionedCall:output:0 ^dropout/StatefulPartitionedCall.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity² Identity_1Identity1pooler_transform/StatefulPartitionedCall:output:0 ^dropout/StatefulPartitionedCall.^embeddings/layer_norm/StatefulPartitionedCall)^pooler_transform/StatefulPartitionedCall+^position_embedding/StatefulPartitionedCall,^transformer/layer_0/StatefulPartitionedCall,^transformer/layer_1/StatefulPartitionedCall-^transformer/layer_10/StatefulPartitionedCall-^transformer/layer_11/StatefulPartitionedCall,^transformer/layer_2/StatefulPartitionedCall,^transformer/layer_3/StatefulPartitionedCall,^transformer/layer_4/StatefulPartitionedCall,^transformer/layer_5/StatefulPartitionedCall,^transformer/layer_6/StatefulPartitionedCall,^transformer/layer_7/StatefulPartitionedCall,^transformer/layer_8/StatefulPartitionedCall,^transformer/layer_9/StatefulPartitionedCall(^type_embeddings/StatefulPartitionedCall(^word_embeddings/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2B dropout/StatefulPartitionedCalldropout/StatefulPartitionedCall2^ -embeddings/layer_norm/StatefulPartitionedCall-embeddings/layer_norm/StatefulPartitionedCall2T (pooler_transform/StatefulPartitionedCall(pooler_transform/StatefulPartitionedCall2X *position_embedding/StatefulPartitionedCall*position_embedding/StatefulPartitionedCall2Z +transformer/layer_0/StatefulPartitionedCall+transformer/layer_0/StatefulPartitionedCall2Z +transformer/layer_1/StatefulPartitionedCall+transformer/layer_1/StatefulPartitionedCall2\ ,transformer/layer_10/StatefulPartitionedCall,transformer/layer_10/StatefulPartitionedCall2\ ,transformer/layer_11/StatefulPartitionedCall,transformer/layer_11/StatefulPartitionedCall2Z +transformer/layer_2/StatefulPartitionedCall+transformer/layer_2/StatefulPartitionedCall2Z +transformer/layer_3/StatefulPartitionedCall+transformer/layer_3/StatefulPartitionedCall2Z +transformer/layer_4/StatefulPartitionedCall+transformer/layer_4/StatefulPartitionedCall2Z +transformer/layer_5/StatefulPartitionedCall+transformer/layer_5/StatefulPartitionedCall2Z +transformer/layer_6/StatefulPartitionedCall+transformer/layer_6/StatefulPartitionedCall2Z +transformer/layer_7/StatefulPartitionedCall+transformer/layer_7/StatefulPartitionedCall2Z +transformer/layer_8/StatefulPartitionedCall+transformer/layer_8/StatefulPartitionedCall2Z +transformer/layer_9/StatefulPartitionedCall+transformer/layer_9/StatefulPartitionedCall2R 'type_embeddings/StatefulPartitionedCall'type_embeddings/StatefulPartitionedCall2R 'word_embeddings/StatefulPartitionedCall'word_embeddings/StatefulPartitionedCall:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ƒ ļ 3__inference_transformer/layer_3_layer_call_fn_26709 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_91102 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : †» £ N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_28119 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_8/ExpandDims/dimõ *self_attention/masked_softmax_8/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_8/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_8/ExpandDims“ %self_attention/masked_softmax_8/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_8/sub/x‚ #self_attention/masked_softmax_8/subSub.self_attention/masked_softmax_8/sub/x:output:03self_attention/masked_softmax_8/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/sub“ %self_attention/masked_softmax_8/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_8/mul/yö #self_attention/masked_softmax_8/mulMul'self_attention/masked_softmax_8/sub:z:0.self_attention/masked_softmax_8/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/mulą #self_attention/masked_softmax_8/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_8/addŅ 'self_attention/masked_softmax_8/SoftmaxSoftmax'self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_8/Softmax— 'self_attention/dropout_24/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_24/dropout/Const† %self_attention/dropout_24/dropout/MulMul1self_attention/masked_softmax_8/Softmax:softmax:00self_attention/dropout_24/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_24/dropout/Mul³ 'self_attention/dropout_24/dropout/ShapeShape1self_attention/masked_softmax_8/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_24/dropout/Shapeœ >self_attention/dropout_24/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_24/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_24/dropout/random_uniform/RandomUniform© 0self_attention/dropout_24/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_24/dropout/GreaterEqual/yĄ .self_attention/dropout_24/dropout/GreaterEqual GreaterEqualGself_attention/dropout_24/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_24/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_24/dropout/GreaterEqualē &self_attention/dropout_24/dropout/CastCast2self_attention/dropout_24/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_24/dropout/Castü 'self_attention/dropout_24/dropout/Mul_1Mul)self_attention/dropout_24/dropout/Mul:z:0*self_attention/dropout_24/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_24/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_24/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_25/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_25/dropout/ConstĆ dropout_25/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_25/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/dropout/Mul‹ dropout_25/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_25/dropout/Shapeć /dropout_25/dropout/random_uniform/RandomUniform RandomUniform!dropout_25/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_25/dropout/random_uniform/RandomUniform‹ !dropout_25/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_25/dropout/GreaterEqual/yų dropout_25/dropout/GreaterEqual GreaterEqual8dropout_25/dropout/random_uniform/RandomUniform:output:0*dropout_25/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_25/dropout/GreaterEqual® dropout_25/dropout/CastCast#dropout_25/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/dropout/Cast“ dropout_25/dropout/Mul_1Muldropout_25/dropout/Mul:z:0dropout_25/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_25/dropout/Mul_1{ addAddV2inputs_0dropout_25/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_8/Pow/yž activation_8/PowPowintermediate/add:z:0activation_8/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Powm activation_8/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_8/mul/xž activation_8/mulMulactivation_8/mul/x:output:0activation_8/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul™ activation_8/addAddV2intermediate/add:z:0activation_8/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/addq activation_8/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_8/mul_1/x¤ activation_8/mul_1Mulactivation_8/mul_1/x:output:0activation_8/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_1† activation_8/TanhTanhactivation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/Tanhq activation_8/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_8/add_1/x§ activation_8/add_1AddV2activation_8/add_1/x:output:0activation_8/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/add_1q activation_8/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_8/mul_2/x¦ activation_8/mul_2Mulactivation_8/mul_2/x:output:0activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_2 activation_8/mul_3Mulintermediate/add:z:0activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_8/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_8/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_26/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_26/dropout/ConstŖ dropout_26/dropout/MulMuloutput/add:z:0!dropout_26/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/dropout/Mulr dropout_26/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_26/dropout/Shapeć /dropout_26/dropout/random_uniform/RandomUniform RandomUniform!dropout_26/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_26/dropout/random_uniform/RandomUniform‹ !dropout_26/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_26/dropout/GreaterEqual/yų dropout_26/dropout/GreaterEqual GreaterEqual8dropout_26/dropout/random_uniform/RandomUniform:output:0*dropout_26/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_26/dropout/GreaterEqual® dropout_26/dropout/CastCast#dropout_26/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/dropout/Cast“ dropout_26/dropout/Mul_1Muldropout_26/dropout/Mul:z:0dropout_26/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_26/dropout/Mul_1¤ add_1AddV2dropout_26/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ¾ŗ £ N__inference_transformer/layer_1_layer_call_and_return_conditional_losses_25942 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_1/ExpandDims/dimõ *self_attention/masked_softmax_1/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_1/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_1/ExpandDims“ %self_attention/masked_softmax_1/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_1/sub/x‚ #self_attention/masked_softmax_1/subSub.self_attention/masked_softmax_1/sub/x:output:03self_attention/masked_softmax_1/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/sub“ %self_attention/masked_softmax_1/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_1/mul/yö #self_attention/masked_softmax_1/mulMul'self_attention/masked_softmax_1/sub:z:0.self_attention/masked_softmax_1/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/mulą #self_attention/masked_softmax_1/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/addŅ 'self_attention/masked_softmax_1/SoftmaxSoftmax'self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_1/Softmax• &self_attention/dropout_3/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2( &self_attention/dropout_3/dropout/Constƒ $self_attention/dropout_3/dropout/MulMul1self_attention/masked_softmax_1/Softmax:softmax:0/self_attention/dropout_3/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout_3/dropout/Mul± &self_attention/dropout_3/dropout/ShapeShape1self_attention/masked_softmax_1/Softmax:softmax:0* T0* _output_shapes :2( &self_attention/dropout_3/dropout/Shape™ =self_attention/dropout_3/dropout/random_uniform/RandomUniform RandomUniform/self_attention/dropout_3/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02? =self_attention/dropout_3/dropout/random_uniform/RandomUniform§ /self_attention/dropout_3/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=21 /self_attention/dropout_3/dropout/GreaterEqual/y¼ -self_attention/dropout_3/dropout/GreaterEqual GreaterEqualFself_attention/dropout_3/dropout/random_uniform/RandomUniform:output:08self_attention/dropout_3/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2/ -self_attention/dropout_3/dropout/GreaterEqualä %self_attention/dropout_3/dropout/CastCast1self_attention/dropout_3/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_3/dropout/Castų &self_attention/dropout_3/dropout/Mul_1Mul(self_attention/dropout_3/dropout/Mul:z:0)self_attention/dropout_3/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_3/dropout/Mul_1’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_3/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addw dropout_4/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_4/dropout/ConstĄ dropout_4/dropout/MulMul'self_attention/attention_output/add:z:0 dropout_4/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/Mul‰ dropout_4/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_4/dropout/Shapeą .dropout_4/dropout/random_uniform/RandomUniform RandomUniform dropout_4/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_4/dropout/random_uniform/RandomUniform‰ dropout_4/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_4/dropout/GreaterEqual/yō dropout_4/dropout/GreaterEqual GreaterEqual7dropout_4/dropout/random_uniform/RandomUniform:output:0)dropout_4/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/GreaterEqual« dropout_4/dropout/CastCast"dropout_4/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/Cast° dropout_4/dropout/Mul_1Muldropout_4/dropout/Mul:z:0dropout_4/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/dropout/Mul_1z addAddV2inputs_0dropout_4/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_1/Pow/yž activation_1/PowPowintermediate/add:z:0activation_1/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Powm activation_1/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_1/mul/xž activation_1/mulMulactivation_1/mul/x:output:0activation_1/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul™ activation_1/addAddV2intermediate/add:z:0activation_1/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/addq activation_1/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_1/mul_1/x¤ activation_1/mul_1Mulactivation_1/mul_1/x:output:0activation_1/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_1† activation_1/TanhTanhactivation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Tanhq activation_1/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_1/add_1/x§ activation_1/add_1AddV2activation_1/add_1/x:output:0activation_1/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/add_1q activation_1/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_1/mul_2/x¦ activation_1/mul_2Mulactivation_1/mul_2/x:output:0activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_2 activation_1/mul_3Mulintermediate/add:z:0activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_1/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addw dropout_5/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_5/dropout/Const§ dropout_5/dropout/MulMuloutput/add:z:0 dropout_5/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/Mulp dropout_5/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_5/dropout/Shapeą .dropout_5/dropout/random_uniform/RandomUniform RandomUniform dropout_5/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_5/dropout/random_uniform/RandomUniform‰ dropout_5/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_5/dropout/GreaterEqual/yō dropout_5/dropout/GreaterEqual GreaterEqual7dropout_5/dropout/random_uniform/RandomUniform:output:0)dropout_5/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/GreaterEqual« dropout_5/dropout/CastCast"dropout_5/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/Cast° dropout_5/dropout/Mul_1Muldropout_5/dropout/Mul:z:0dropout_5/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/dropout/Mul_1£ add_1AddV2dropout_5/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ė ³ K__inference_pooler_transform_layer_call_and_return_conditional_losses_29274 inputs" matmul_readvariableop_resource# biasadd_readvariableop_resource identityˆ MatMul/ReadVariableOpReadVariableOpmatmul_readvariableop_resource* _output_shapes : €€* dtype02 MatMul/ReadVariableOpt MatMulMatMulinputsMatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2 MatMul BiasAdd/ReadVariableOpReadVariableOpbiasadd_readvariableop_resource* _output_shapes :€* dtype02 BiasAdd/ReadVariableOp‚ BiasAddBiasAddMatMul:product:0BiasAdd/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2 BiasAddY TanhTanhBiasAdd:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Tanh] IdentityIdentityTanh:y:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*/ _input_shapes : ’’’’’’’’’€:::PL ( _output_shapes : ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes ::  _output_shapes : Ģ ˆ L__inference_position_embedding_layer_call_and_return_conditional_losses_7663 inputs readvariableop_resource identityˆD ShapeShapeinputs* T0* _output_shapes :2 Shapet strided_slice/stackConst* _output_shapes :* dtype0* value B :2 strided_slice/stackx strided_slice/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_1x strided_slice/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_2ā strided_slice StridedSliceShape:output:0strided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slicex strided_slice_1/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_1/stack| strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_1| strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_2ģ strided_slice_1 StridedSliceShape:output:0strided_slice_1/stack:output:0 strided_slice_1/stack_1:output:0 strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_1z ReadVariableOpReadVariableOpreadvariableop_resource* _output_shapes : €€* dtype02 ReadVariableOp strided_slice_2/stackConst* _output_shapes :* dtype0* valueB"2 strided_slice_2/stackx strided_slice_2/stack_1/1Const* _output_shapes :* dtype0* value B:2 strided_slice_2/stack_1/1¦ strided_slice_2/stack_1Packstrided_slice_1:output:0"strided_slice_2/stack_1/1:output:0* N* T0* _output_shapes :2 strided_slice_2/stack_1ƒ strided_slice_2/stack_2Const* _output_shapes :* dtype0* valueB"2 strided_slice_2/stack_2 strided_slice_2 StridedSliceReadVariableOp:value:0strided_slice_2/stack:output:0 strided_slice_2/stack_1:output:0 strided_slice_2/stack_2:output:0* Index0* T0*( _output_shapes : ’’’’’’’’’€* begin_mask* end_mask2 strided_slice_2m BroadcastTo/shape/2Const* _output_shapes :* dtype0* value B:€2 BroadcastTo/shape/2¬ BroadcastTo/shapePackstrided_slice:output:0strided_slice_1:output:0BroadcastTo/shape/2:output:0* N* T0* _output_shapes :2 BroadcastTo/shapeŸ BroadcastTo BroadcastTostrided_slice_2:output:0BroadcastTo/shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 BroadcastTov IdentityIdentityBroadcastTo:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*8 _input_shapes' %: ’’’’’’’’’ ’’’’’’’’’€::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes : % z N__inference_self_attention_mask_layer_call_and_return_conditional_losses_25497 inputs_0 inputs_1 identityF ShapeShapeinputs_0* T0* _output_shapes :2 Shapet strided_slice/stackConst* _output_shapes :* dtype0* value B :2 strided_slice/stackx strided_slice/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_1x strided_slice/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_2ā strided_slice StridedSliceShape:output:0strided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slicex strided_slice_1/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_1/stack| strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_1| strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_2ģ strided_slice_1 StridedSliceShape:output:0strided_slice_1/stack:output:0 strided_slice_1/stack_1:output:0 strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_1J Shape_1Shapeinputs_1* T0* _output_shapes :2 Shape_1x strided_slice_2/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_2/stack| strided_slice_2/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_2/stack_1| strided_slice_2/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_2/stack_2ī strided_slice_2 StridedSliceShape_1:output:0strided_slice_2/stack:output:0 strided_slice_2/stack_1:output:0 strided_slice_2/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_2x strided_slice_3/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_3/stack| strided_slice_3/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_3/stack_1| strided_slice_3/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_3/stack_2ī strided_slice_3 StridedSliceShape_1:output:0strided_slice_3/stack:output:0 strided_slice_3/stack_1:output:0 strided_slice_3/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_3d Reshape/shape/1Const* _output_shapes :* dtype0* value B:2 Reshape/shape/1  Reshape/shapePackstrided_slice:output:0Reshape/shape/1:output:0strided_slice_3:output:0* N* T0* _output_shapes :2 Reshape/shape~ ReshapeReshapeinputs_1Reshape/shape:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 Reshapet CastCastReshape:output:0* DstT0* SrcT0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 Castn ones/mulMulstrided_slice:output:0strided_slice_1:output:0* T0* _output_shapes :2 ones/mul^ ones/mul_1/yConst* _output_shapes :* dtype0* value B:2 ones/mul_1/ye ones/mul_1Mul ones/mul:z:0ones/mul_1/y:output:0* T0* _output_shapes :2 ones/mul_1] ones/Less/yConst* _output_shapes :* dtype0* value B:č2 ones/Less/ye ones/LessLessones/mul_1:z:0ones/Less/y:output:0* T0* _output_shapes :2 ones/Less` ones/packed/2Const* _output_shapes :* dtype0* value B:2 ones/packed/2š ones/packedPackstrided_slice:output:0strided_slice_1:output:0ones/packed/2:output:0* N* T0* _output_shapes :2 ones/packed] ones/ConstConst* _output_shapes :* dtype0* value B *€?2 ones/Const~ onesFillones/packed:output:0ones/Const:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 onesr mulMul ones:output:0Cast:y:0* T0*= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2 mulq IdentityIdentitymul:z:0* T0*= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2 Identity" identityIdentity:output:0*P _input_shapes? =: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1  u /__inference_type_embeddings_layer_call_fn_25387 inputs unknown identityˆ¢StatefulPartitionedCall÷ StatefulPartitionedCallStatefulPartitionedCallinputsunknown* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*# _read_only_resource_inputs ** config_proto  CPU  GPU2J8*R fMRK I__inference_type_embeddings_layer_call_and_return_conditional_losses_76932 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*3 _input_shapes" : ’’’’’’’’’ ’’’’’’’’’:22 StatefulPartitionedCallStatefulPartitionedCall:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes : ‚ _ 3__inference_self_attention_mask_layer_call_fn_25503 inputs_0 inputs_1 identityė PartitionedCallPartitionedCallinputs_0inputs_1* Tin 2* Tout 2* _collective_manager_ids *= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*V fQRO M__inference_self_attention_mask_layer_call_and_return_conditional_losses_78282 PartitionedCall‚ IdentityIdentityPartitionedCall:output:0* T0*= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2 Identity" identityIdentity:output:0*P _input_shapes? =: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1 „ ļ 3__inference_transformer/layer_7_layer_call_fn_27991 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_104942 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ßŗ   M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_9003 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_3/ExpandDims/dimõ *self_attention/masked_softmax_3/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_3/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_3/ExpandDims“ %self_attention/masked_softmax_3/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_3/sub/x‚ #self_attention/masked_softmax_3/subSub.self_attention/masked_softmax_3/sub/x:output:03self_attention/masked_softmax_3/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/sub“ %self_attention/masked_softmax_3/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_3/mul/yö #self_attention/masked_softmax_3/mulMul'self_attention/masked_softmax_3/sub:z:0.self_attention/masked_softmax_3/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/mulą #self_attention/masked_softmax_3/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/addŅ 'self_attention/masked_softmax_3/SoftmaxSoftmax'self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_3/Softmax• &self_attention/dropout_9/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2( &self_attention/dropout_9/dropout/Constƒ $self_attention/dropout_9/dropout/MulMul1self_attention/masked_softmax_3/Softmax:softmax:0/self_attention/dropout_9/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout_9/dropout/Mul± &self_attention/dropout_9/dropout/ShapeShape1self_attention/masked_softmax_3/Softmax:softmax:0* T0* _output_shapes :2( &self_attention/dropout_9/dropout/Shape™ =self_attention/dropout_9/dropout/random_uniform/RandomUniform RandomUniform/self_attention/dropout_9/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02? =self_attention/dropout_9/dropout/random_uniform/RandomUniform§ /self_attention/dropout_9/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=21 /self_attention/dropout_9/dropout/GreaterEqual/y¼ -self_attention/dropout_9/dropout/GreaterEqual GreaterEqualFself_attention/dropout_9/dropout/random_uniform/RandomUniform:output:08self_attention/dropout_9/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2/ -self_attention/dropout_9/dropout/GreaterEqualä %self_attention/dropout_9/dropout/CastCast1self_attention/dropout_9/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_9/dropout/Castų &self_attention/dropout_9/dropout/Mul_1Mul(self_attention/dropout_9/dropout/Mul:z:0)self_attention/dropout_9/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_9/dropout/Mul_1’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_9/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_10/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_10/dropout/ConstĆ dropout_10/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_10/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/dropout/Mul‹ dropout_10/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_10/dropout/Shapeć /dropout_10/dropout/random_uniform/RandomUniform RandomUniform!dropout_10/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_10/dropout/random_uniform/RandomUniform‹ !dropout_10/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_10/dropout/GreaterEqual/yų dropout_10/dropout/GreaterEqual GreaterEqual8dropout_10/dropout/random_uniform/RandomUniform:output:0*dropout_10/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_10/dropout/GreaterEqual® dropout_10/dropout/CastCast#dropout_10/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/dropout/Cast“ dropout_10/dropout/Mul_1Muldropout_10/dropout/Mul:z:0dropout_10/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/dropout/Mul_1y addAddV2inputsdropout_10/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_3/Pow/yž activation_3/PowPowintermediate/add:z:0activation_3/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Powm activation_3/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_3/mul/xž activation_3/mulMulactivation_3/mul/x:output:0activation_3/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul™ activation_3/addAddV2intermediate/add:z:0activation_3/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/addq activation_3/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_3/mul_1/x¤ activation_3/mul_1Mulactivation_3/mul_1/x:output:0activation_3/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_1† activation_3/TanhTanhactivation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Tanhq activation_3/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_3/add_1/x§ activation_3/add_1AddV2activation_3/add_1/x:output:0activation_3/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/add_1q activation_3/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_3/mul_2/x¦ activation_3/mul_2Mulactivation_3/mul_2/x:output:0activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_2 activation_3/mul_3Mulintermediate/add:z:0activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_3/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_11/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_11/dropout/ConstŖ dropout_11/dropout/MulMuloutput/add:z:0!dropout_11/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/dropout/Mulr dropout_11/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_11/dropout/Shapeć /dropout_11/dropout/random_uniform/RandomUniform RandomUniform!dropout_11/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_11/dropout/random_uniform/RandomUniform‹ !dropout_11/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_11/dropout/GreaterEqual/yų dropout_11/dropout/GreaterEqual GreaterEqual8dropout_11/dropout/random_uniform/RandomUniform:output:0*dropout_11/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_11/dropout/GreaterEqual® dropout_11/dropout/CastCast#dropout_11/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/dropout/Cast“ dropout_11/dropout/Mul_1Muldropout_11/dropout/Mul:z:0dropout_11/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/dropout/Mul_1¤ add_1AddV2dropout_11/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ž ] A__inference_lambda_layer_call_and_return_conditional_losses_11998 inputs identity strided_slice/stackConst* _output_shapes :* dtype0*! valueB" 2 strided_slice/stackƒ strided_slice/stack_1Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_1ƒ strided_slice/stack_2Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_2ƒ strided_slice StridedSliceinputsstrided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* begin_mask* end_mask2 strided_slice SqueezeSqueezestrided_slice:output:0* T0*( _output_shapes : ’’’’’’’’’€* squeeze_dims 2 Squeezee IdentityIdentitySqueeze:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs ՟ Ł1 G__inference_functional_1_layer_call_and_return_conditional_losses_16183 inputs inputs_1 inputs_2 transformer_encoder_15781 transformer_encoder_15783 transformer_encoder_15785 transformer_encoder_15787 transformer_encoder_15789 transformer_encoder_15791 transformer_encoder_15793 transformer_encoder_15795 transformer_encoder_15797 transformer_encoder_15799 transformer_encoder_15801 transformer_encoder_15803 transformer_encoder_15805 transformer_encoder_15807 transformer_encoder_15809 transformer_encoder_15811 transformer_encoder_15813 transformer_encoder_15815 transformer_encoder_15817 transformer_encoder_15819 transformer_encoder_15821 transformer_encoder_15823 transformer_encoder_15825 transformer_encoder_15827 transformer_encoder_15829 transformer_encoder_15831 transformer_encoder_15833 transformer_encoder_15835 transformer_encoder_15837 transformer_encoder_15839 transformer_encoder_15841 transformer_encoder_15843 transformer_encoder_15845 transformer_encoder_15847 transformer_encoder_15849 transformer_encoder_15851 transformer_encoder_15853 transformer_encoder_15855 transformer_encoder_15857 transformer_encoder_15859 transformer_encoder_15861 transformer_encoder_15863 transformer_encoder_15865 transformer_encoder_15867 transformer_encoder_15869 transformer_encoder_15871 transformer_encoder_15873 transformer_encoder_15875 transformer_encoder_15877 transformer_encoder_15879 transformer_encoder_15881 transformer_encoder_15883 transformer_encoder_15885 transformer_encoder_15887 transformer_encoder_15889 transformer_encoder_15891 transformer_encoder_15893 transformer_encoder_15895 transformer_encoder_15897 transformer_encoder_15899 transformer_encoder_15901 transformer_encoder_15903 transformer_encoder_15905 transformer_encoder_15907 transformer_encoder_15909 transformer_encoder_15911 transformer_encoder_15913 transformer_encoder_15915 transformer_encoder_15917 transformer_encoder_15919 transformer_encoder_15921 transformer_encoder_15923 transformer_encoder_15925 transformer_encoder_15927 transformer_encoder_15929 transformer_encoder_15931 transformer_encoder_15933 transformer_encoder_15935 transformer_encoder_15937 transformer_encoder_15939 transformer_encoder_15941 transformer_encoder_15943 transformer_encoder_15945 transformer_encoder_15947 transformer_encoder_15949 transformer_encoder_15951 transformer_encoder_15953 transformer_encoder_15955 transformer_encoder_15957 transformer_encoder_15959 transformer_encoder_15961 transformer_encoder_15963 transformer_encoder_15965 transformer_encoder_15967 transformer_encoder_15969 transformer_encoder_15971 transformer_encoder_15973 transformer_encoder_15975 transformer_encoder_15977 transformer_encoder_15979 transformer_encoder_15981 transformer_encoder_15983 transformer_encoder_15985 transformer_encoder_15987 transformer_encoder_15989 transformer_encoder_15991 transformer_encoder_15993 transformer_encoder_15995 transformer_encoder_15997 transformer_encoder_15999 transformer_encoder_16001 transformer_encoder_16003 transformer_encoder_16005 transformer_encoder_16007 transformer_encoder_16009 transformer_encoder_16011 transformer_encoder_16013 transformer_encoder_16015 transformer_encoder_16017 transformer_encoder_16019 transformer_encoder_16021 transformer_encoder_16023 transformer_encoder_16025 transformer_encoder_16027 transformer_encoder_16029 transformer_encoder_16031 transformer_encoder_16033 transformer_encoder_16035 transformer_encoder_16037 transformer_encoder_16039 transformer_encoder_16041 transformer_encoder_16043 transformer_encoder_16045 transformer_encoder_16047 transformer_encoder_16049 transformer_encoder_16051 transformer_encoder_16053 transformer_encoder_16055 transformer_encoder_16057 transformer_encoder_16059 transformer_encoder_16061 transformer_encoder_16063 transformer_encoder_16065 transformer_encoder_16067 transformer_encoder_16069 transformer_encoder_16071 transformer_encoder_16073 transformer_encoder_16075 transformer_encoder_16077 transformer_encoder_16079 transformer_encoder_16081 transformer_encoder_16083 transformer_encoder_16085 transformer_encoder_16087 transformer_encoder_16089 transformer_encoder_16091 transformer_encoder_16093 transformer_encoder_16095 transformer_encoder_16097 transformer_encoder_16099 transformer_encoder_16101 transformer_encoder_16103 transformer_encoder_16105 transformer_encoder_16107 transformer_encoder_16109 transformer_encoder_16111 transformer_encoder_16113 transformer_encoder_16115 transformer_encoder_16117 transformer_encoder_16119 transformer_encoder_16121 transformer_encoder_16123 transformer_encoder_16125 transformer_encoder_16127 transformer_encoder_16129 transformer_encoder_16131 transformer_encoder_16133 transformer_encoder_16135 transformer_encoder_16137 transformer_encoder_16139 transformer_encoder_16141 transformer_encoder_16143 transformer_encoder_16145 transformer_encoder_16147 transformer_encoder_16149 transformer_encoder_16151 transformer_encoder_16153 transformer_encoder_16155 transformer_encoder_16157 transformer_encoder_16159 transformer_encoder_16161 transformer_encoder_16163 transformer_encoder_16165 transformer_encoder_16167 transformer_encoder_16169 transformer_encoder_16171 transformer_encoder_16173 transformer_encoder_16175 transformer_encoder_16177 identity identity_1ˆ¢+transformer_encoder/StatefulPartitionedCall”1 +transformer_encoder/StatefulPartitionedCallStatefulPartitionedCallinputsinputs_1inputs_2transformer_encoder_15781transformer_encoder_15783transformer_encoder_15785transformer_encoder_15787transformer_encoder_15789transformer_encoder_15791transformer_encoder_15793transformer_encoder_15795transformer_encoder_15797transformer_encoder_15799transformer_encoder_15801transformer_encoder_15803transformer_encoder_15805transformer_encoder_15807transformer_encoder_15809transformer_encoder_15811transformer_encoder_15813transformer_encoder_15815transformer_encoder_15817transformer_encoder_15819transformer_encoder_15821transformer_encoder_15823transformer_encoder_15825transformer_encoder_15827transformer_encoder_15829transformer_encoder_15831transformer_encoder_15833transformer_encoder_15835transformer_encoder_15837transformer_encoder_15839transformer_encoder_15841transformer_encoder_15843transformer_encoder_15845transformer_encoder_15847transformer_encoder_15849transformer_encoder_15851transformer_encoder_15853transformer_encoder_15855transformer_encoder_15857transformer_encoder_15859transformer_encoder_15861transformer_encoder_15863transformer_encoder_15865transformer_encoder_15867transformer_encoder_15869transformer_encoder_15871transformer_encoder_15873transformer_encoder_15875transformer_encoder_15877transformer_encoder_15879transformer_encoder_15881transformer_encoder_15883transformer_encoder_15885transformer_encoder_15887transformer_encoder_15889transformer_encoder_15891transformer_encoder_15893transformer_encoder_15895transformer_encoder_15897transformer_encoder_15899transformer_encoder_15901transformer_encoder_15903transformer_encoder_15905transformer_encoder_15907transformer_encoder_15909transformer_encoder_15911transformer_encoder_15913transformer_encoder_15915transformer_encoder_15917transformer_encoder_15919transformer_encoder_15921transformer_encoder_15923transformer_encoder_15925transformer_encoder_15927transformer_encoder_15929transformer_encoder_15931transformer_encoder_15933transformer_encoder_15935transformer_encoder_15937transformer_encoder_15939transformer_encoder_15941transformer_encoder_15943transformer_encoder_15945transformer_encoder_15947transformer_encoder_15949transformer_encoder_15951transformer_encoder_15953transformer_encoder_15955transformer_encoder_15957transformer_encoder_15959transformer_encoder_15961transformer_encoder_15963transformer_encoder_15965transformer_encoder_15967transformer_encoder_15969transformer_encoder_15971transformer_encoder_15973transformer_encoder_15975transformer_encoder_15977transformer_encoder_15979transformer_encoder_15981transformer_encoder_15983transformer_encoder_15985transformer_encoder_15987transformer_encoder_15989transformer_encoder_15991transformer_encoder_15993transformer_encoder_15995transformer_encoder_15997transformer_encoder_15999transformer_encoder_16001transformer_encoder_16003transformer_encoder_16005transformer_encoder_16007transformer_encoder_16009transformer_encoder_16011transformer_encoder_16013transformer_encoder_16015transformer_encoder_16017transformer_encoder_16019transformer_encoder_16021transformer_encoder_16023transformer_encoder_16025transformer_encoder_16027transformer_encoder_16029transformer_encoder_16031transformer_encoder_16033transformer_encoder_16035transformer_encoder_16037transformer_encoder_16039transformer_encoder_16041transformer_encoder_16043transformer_encoder_16045transformer_encoder_16047transformer_encoder_16049transformer_encoder_16051transformer_encoder_16053transformer_encoder_16055transformer_encoder_16057transformer_encoder_16059transformer_encoder_16061transformer_encoder_16063transformer_encoder_16065transformer_encoder_16067transformer_encoder_16069transformer_encoder_16071transformer_encoder_16073transformer_encoder_16075transformer_encoder_16077transformer_encoder_16079transformer_encoder_16081transformer_encoder_16083transformer_encoder_16085transformer_encoder_16087transformer_encoder_16089transformer_encoder_16091transformer_encoder_16093transformer_encoder_16095transformer_encoder_16097transformer_encoder_16099transformer_encoder_16101transformer_encoder_16103transformer_encoder_16105transformer_encoder_16107transformer_encoder_16109transformer_encoder_16111transformer_encoder_16113transformer_encoder_16115transformer_encoder_16117transformer_encoder_16119transformer_encoder_16121transformer_encoder_16123transformer_encoder_16125transformer_encoder_16127transformer_encoder_16129transformer_encoder_16131transformer_encoder_16133transformer_encoder_16135transformer_encoder_16137transformer_encoder_16139transformer_encoder_16141transformer_encoder_16143transformer_encoder_16145transformer_encoder_16147transformer_encoder_16149transformer_encoder_16151transformer_encoder_16153transformer_encoder_16155transformer_encoder_16157transformer_encoder_16159transformer_encoder_16161transformer_encoder_16163transformer_encoder_16165transformer_encoder_16167transformer_encoder_16169transformer_encoder_16171transformer_encoder_16173transformer_encoder_16175transformer_encoder_16177*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_129062- +transformer_encoder/StatefulPartitionedCall· IdentityIdentity4transformer_encoder/StatefulPartitionedCall:output:1,^transformer_encoder/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 IdentityČ Identity_1Identity4transformer_encoder/StatefulPartitionedCall:output:0,^transformer_encoder/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2Z +transformer_encoder/StatefulPartitionedCall+transformer_encoder/StatefulPartitionedCall:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ƒ ļ 3__inference_transformer/layer_2_layer_call_fn_26398 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_87642 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ń a B__inference_dropout_layer_call_and_return_conditional_losses_25445 inputs identityˆc dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout/Const dropout/MulMulinputsdropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/MulT dropout/ShapeShapeinputs* T0* _output_shapes :2 dropout/ShapeĀ $dropout/random_uniform/RandomUniform RandomUniformdropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02& $dropout/random_uniform/RandomUniformu dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2 dropout/GreaterEqual/yĢ dropout/GreaterEqual GreaterEqual-dropout/random_uniform/RandomUniform:output:0dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/GreaterEqual dropout/CastCastdropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/Castˆ dropout/Mul_1Muldropout/Mul:z:0dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/Mul_1s IdentityIdentitydropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs ė ³ K__inference_pooler_transform_layer_call_and_return_conditional_losses_12031 inputs" matmul_readvariableop_resource# biasadd_readvariableop_resource identityˆ MatMul/ReadVariableOpReadVariableOpmatmul_readvariableop_resource* _output_shapes : €€* dtype02 MatMul/ReadVariableOpt MatMulMatMulinputsMatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2 MatMul BiasAdd/ReadVariableOpReadVariableOpbiasadd_readvariableop_resource* _output_shapes :€* dtype02 BiasAdd/ReadVariableOp‚ BiasAddBiasAddMatMul:product:0BiasAdd/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2 BiasAddY TanhTanhBiasAdd:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Tanh] IdentityIdentityTanh:y:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*/ _input_shapes : ’’’’’’’’’€:::PL ( _output_shapes : ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_3_layer_call_fn_26747 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_91102 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ī ` '__inference_dropout_layer_call_fn_25455 inputs identityˆ¢StatefulPartitionedCallā StatefulPartitionedCallStatefulPartitionedCallinputs* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_dropout_layer_call_and_return_conditional_losses_77732 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€22 StatefulPartitionedCallStatefulPartitionedCall:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs Ž ] A__inference_lambda_layer_call_and_return_conditional_losses_29244 inputs identity strided_slice/stackConst* _output_shapes :* dtype0*! valueB" 2 strided_slice/stackƒ strided_slice/stack_1Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_1ƒ strided_slice/stack_2Const* _output_shapes :* dtype0*! valueB" 2 strided_slice/stack_2ƒ strided_slice StridedSliceinputsstrided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* begin_mask* end_mask2 strided_slice SqueezeSqueezestrided_slice:output:0* T0*( _output_shapes : ’’’’’’’’’€* squeeze_dims 2 Squeezee IdentityIdentitySqueeze:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs š ” N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_10494 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_7/ExpandDims/dimõ *self_attention/masked_softmax_7/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_7/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_7/ExpandDims“ %self_attention/masked_softmax_7/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_7/sub/x‚ #self_attention/masked_softmax_7/subSub.self_attention/masked_softmax_7/sub/x:output:03self_attention/masked_softmax_7/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/sub“ %self_attention/masked_softmax_7/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_7/mul/yö #self_attention/masked_softmax_7/mulMul'self_attention/masked_softmax_7/sub:z:0.self_attention/masked_softmax_7/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/mulą #self_attention/masked_softmax_7/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/addŅ 'self_attention/masked_softmax_7/SoftmaxSoftmax'self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_7/SoftmaxÓ "self_attention/dropout_21/IdentityIdentity1self_attention/masked_softmax_7/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_21/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_21/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_22/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/Identityy addAddV2inputsdropout_22/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_7/Pow/yž activation_7/PowPowintermediate/add:z:0activation_7/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Powm activation_7/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_7/mul/xž activation_7/mulMulactivation_7/mul/x:output:0activation_7/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul™ activation_7/addAddV2intermediate/add:z:0activation_7/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/addq activation_7/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_7/mul_1/x¤ activation_7/mul_1Mulactivation_7/mul_1/x:output:0activation_7/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_1† activation_7/TanhTanhactivation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Tanhq activation_7/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_7/add_1/x§ activation_7/add_1AddV2activation_7/add_1/x:output:0activation_7/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/add_1q activation_7/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_7/mul_2/x¦ activation_7/mul_2Mulactivation_7/mul_2/x:output:0activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_2 activation_7/mul_3Mulintermediate/add:z:0activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_7/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_23/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/Identity¤ add_1AddV2dropout_23/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : š   M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_9802 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_5/ExpandDims/dimõ *self_attention/masked_softmax_5/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_5/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_5/ExpandDims“ %self_attention/masked_softmax_5/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_5/sub/x‚ #self_attention/masked_softmax_5/subSub.self_attention/masked_softmax_5/sub/x:output:03self_attention/masked_softmax_5/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/sub“ %self_attention/masked_softmax_5/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_5/mul/yö #self_attention/masked_softmax_5/mulMul'self_attention/masked_softmax_5/sub:z:0.self_attention/masked_softmax_5/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/mulą #self_attention/masked_softmax_5/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/addŅ 'self_attention/masked_softmax_5/SoftmaxSoftmax'self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_5/SoftmaxÓ "self_attention/dropout_15/IdentityIdentity1self_attention/masked_softmax_5/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_15/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_15/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_16/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/Identityy addAddV2inputsdropout_16/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_5/Pow/yž activation_5/PowPowintermediate/add:z:0activation_5/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Powm activation_5/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_5/mul/xž activation_5/mulMulactivation_5/mul/x:output:0activation_5/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul™ activation_5/addAddV2intermediate/add:z:0activation_5/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/addq activation_5/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_5/mul_1/x¤ activation_5/mul_1Mulactivation_5/mul_1/x:output:0activation_5/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_1† activation_5/TanhTanhactivation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Tanhq activation_5/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_5/add_1/x§ activation_5/add_1AddV2activation_5/add_1/x:output:0activation_5/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/add_1q activation_5/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_5/mul_2/x¦ activation_5/mul_2Mulactivation_5/mul_2/x:output:0activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_2 activation_5/mul_3Mulintermediate/add:z:0activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_5/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_17/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/Identity¤ add_1AddV2dropout_17/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ’€ õy N__inference_transformer_encoder_layer_call_and_return_conditional_losses_23158 inputs_0 inputs_1 inputs_2# word_embeddings_gather_resource. *position_embedding_readvariableop_resource2 .type_embeddings_matmul_readvariableop_resource? ;embeddings_layer_norm_batchnorm_mul_readvariableop_resource; 7embeddings_layer_norm_batchnorm_readvariableop_resourceR Ntransformer_layer_0_self_attention_query_einsum_einsum_readvariableop_resourceH Dtransformer_layer_0_self_attention_query_add_readvariableop_resourceP Ltransformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resourceF Btransformer_layer_0_self_attention_key_add_readvariableop_resourceR Ntransformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resourceH Dtransformer_layer_0_self_attention_value_add_readvariableop_resource] Ytransformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resourceS Otransformer_layer_0_self_attention_attention_output_add_readvariableop_resourceW Stransformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceS Otransformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resourceJ Ftransformer_layer_0_intermediate_einsum_einsum_readvariableop_resource@ abde28 6transformer/layer_0/self_attention/query/einsum/Einsum’ ;transformer/layer_0/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_0_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_0/self_attention/query/add/ReadVariableOp® ,transformer/layer_0/self_attention/query/addAddV2?transformer/layer_0/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_0/self_attention/query/addœ Ctransformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpĒ 4transformer/layer_0/self_attention/key/einsum/EinsumEinsumdropout/dropout/Mul_1:z:0Ktransformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_0/self_attention/key/einsum/Einsumł 9transformer/layer_0/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_0_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_0/self_attention/key/add/ReadVariableOp¦ *transformer/layer_0/self_attention/key/addAddV2=transformer/layer_0/self_attention/key/einsum/Einsum:output:0Atransformer/layer_0/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_0/self_attention/key/add¢ Etransformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpĶ 6transformer/layer_0/self_attention/value/einsum/EinsumEinsumdropout/dropout/Mul_1:z:0Mtransformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_0/self_attention/value/einsum/Einsum’ ;transformer/layer_0/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_0_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_0/self_attention/value/add/ReadVariableOp® ,transformer/layer_0/self_attention/value/addAddV2?transformer/layer_0/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_0/self_attention/value/addÄ 0transformer/layer_0/self_attention/einsum/EinsumEinsum.transformer/layer_0/self_attention/key/add:z:00transformer/layer_0/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_0/self_attention/einsum/Einsum™ (transformer/layer_0/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_0/self_attention/Mul/y‘ &transformer/layer_0/self_attention/MulMul9transformer/layer_0/self_attention/einsum/Einsum:output:01transformer/layer_0/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_0/self_attention/MulĪ @transformer/layer_0/self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2B @transformer/layer_0/self_attention/masked_softmax/ExpandDims/dim¾ abcd24 2transformer/layer_0/self_attention/einsum_1/EinsumĆ Ptransformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_0/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_0/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_0/self_attention/attention_output/einsum/Einsum Ftransformer/layer_0/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_0_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_0/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_0/self_attention/attention_output/addAddV2Jtransformer/layer_0/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_0/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_0/self_attention/attention_output/addŸ +transformer/layer_0/dropout_1/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2- +transformer/layer_0/dropout_1/dropout/Const )transformer/layer_0/dropout_1/dropout/MulMul;transformer/layer_0/self_attention/attention_output/add:z:04transformer/layer_0/dropout_1/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer/layer_0/dropout_1/dropout/MulÅ +transformer/layer_0/dropout_1/dropout/ShapeShape;transformer/layer_0/self_attention/attention_output/add:z:0* T0* _output_shapes :2- +transformer/layer_0/dropout_1/dropout/Shapeœ Btransformer/layer_0/dropout_1/dropout/random_uniform/RandomUniform RandomUniform4transformer/layer_0/dropout_1/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02D Btransformer/layer_0/dropout_1/dropout/random_uniform/RandomUniform± 4transformer/layer_0/dropout_1/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=26 4transformer/layer_0/dropout_1/dropout/GreaterEqual/yÄ 2transformer/layer_0/dropout_1/dropout/GreaterEqual GreaterEqualKtransformer/layer_0/dropout_1/dropout/random_uniform/RandomUniform:output:0=transformer/layer_0/dropout_1/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer/layer_0/dropout_1/dropout/GreaterEqualē *transformer/layer_0/dropout_1/dropout/CastCast6transformer/layer_0/dropout_1/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_0/dropout_1/dropout/Cast€ +transformer/layer_0/dropout_1/dropout/Mul_1Mul-transformer/layer_0/dropout_1/dropout/Mul:z:0.transformer/layer_0/dropout_1/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_0/dropout_1/dropout/Mul_1Ē transformer/layer_0/addAddV2dropout/dropout/Mul_1:z:0/transformer/layer_0/dropout_1/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_0/addę Ltransformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_0/self_attention_layer_norm/moments/meanMeantransformer/layer_0/add:z:0Utransformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_0/self_attention_layer_norm/moments/meanœ Btransformer/layer_0/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_0/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_0/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_0/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_0/add:z:0Ktransformer/layer_0/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_0/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_0/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_0/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_0/self_attention_layer_norm/moments/varianceĆ =transformer/layer_0/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_0/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_0/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_0/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_0/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_0/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_0/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_0/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_0/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_0/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_0/add:z:0?transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_0/self_attention_layer_norm/moments/mean:output:0?transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_0/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_0/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_0/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_0/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_0/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_0/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_0/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_0_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_0/intermediate/einsum/EinsumEinsumAtransformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_0/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_0/intermediate/einsum/Einsumä 3transformer/layer_0/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_0/output/einsum/EinsumŅ -transformer/layer_0/output/add/ReadVariableOpReadVariableOp6transformer_layer_0_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_0/output/add/ReadVariableOpó transformer/layer_0/output/addAddV21transformer/layer_0/output/einsum/Einsum:output:05transformer/layer_0/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_0/output/addŸ +transformer/layer_0/dropout_2/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2- +transformer/layer_0/dropout_2/dropout/Const÷ )transformer/layer_0/dropout_2/dropout/MulMul"transformer/layer_0/output/add:z:04transformer/layer_0/dropout_2/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer/layer_0/dropout_2/dropout/Mul¬ +transformer/layer_0/dropout_2/dropout/ShapeShape"transformer/layer_0/output/add:z:0* T0* _output_shapes :2- +transformer/layer_0/dropout_2/dropout/Shapeœ Btransformer/layer_0/dropout_2/dropout/random_uniform/RandomUniform RandomUniform4transformer/layer_0/dropout_2/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02D Btransformer/layer_0/dropout_2/dropout/random_uniform/RandomUniform± 4transformer/layer_0/dropout_2/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=26 4transformer/layer_0/dropout_2/dropout/GreaterEqual/yÄ 2transformer/layer_0/dropout_2/dropout/GreaterEqual GreaterEqualKtransformer/layer_0/dropout_2/dropout/random_uniform/RandomUniform:output:0=transformer/layer_0/dropout_2/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer/layer_0/dropout_2/dropout/GreaterEqualē *transformer/layer_0/dropout_2/dropout/CastCast6transformer/layer_0/dropout_2/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_0/dropout_2/dropout/Cast€ +transformer/layer_0/dropout_2/dropout/Mul_1Mul-transformer/layer_0/dropout_2/dropout/Mul:z:0.transformer/layer_0/dropout_2/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_0/dropout_2/dropout/Mul_1ó transformer/layer_0/add_1AddV2/transformer/layer_0/dropout_2/dropout/Mul_1:z:0Atransformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_0/add_1Ö Dtransformer/layer_0/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_0/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_0/output_layer_norm/moments/meanMeantransformer/layer_0/add_1:z:0Mtransformer/layer_0/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_0/output_layer_norm/moments/mean„ :transformer/layer_0/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_0/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_0/output_layer_norm/moments/StopGradient» ?transformer/layer_0/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_0/add_1:z:0Ctransformer/layer_0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_0/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_0/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_0/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_0/output_layer_norm/moments/varianceMeanCtransformer/layer_0/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_0/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_0/output_layer_norm/moments/variance³ 5transformer/layer_0/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_0/output_layer_norm/batchnorm/add/y³ 3transformer/layer_0/output_layer_norm/batchnorm/addAddV2?transformer/layer_0/output_layer_norm/moments/variance:output:0>transformer/layer_0/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_0/output_layer_norm/batchnorm/addļ 5transformer/layer_0/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_0/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_0/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_0/output_layer_norm/batchnorm/mulMul9transformer/layer_0/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_0/output_layer_norm/batchnorm/mul 5transformer/layer_0/output_layer_norm/batchnorm/mul_1Multransformer/layer_0/add_1:z:07transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_0/output_layer_norm/batchnorm/mul_1« 5transformer/layer_0/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_0/output_layer_norm/moments/mean:output:07transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_0/output_layer_norm/batchnorm/mul_2… >transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_0_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_0/output_layer_norm/batchnorm/subSubFtransformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_0/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_0/output_layer_norm/batchnorm/sub« 5transformer/layer_0/output_layer_norm/batchnorm/add_1AddV29transformer/layer_0/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_0/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_0/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_1/self_attention/query/einsum/EinsumEinsum9transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_1/self_attention/query/einsum/Einsum’ ;transformer/layer_1/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_1_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_1/self_attention/query/add/ReadVariableOp® ,transformer/layer_1/self_attention/query/addAddV2?transformer/layer_1/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_1/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_1/self_attention/query/addœ Ctransformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_1/self_attention/key/einsum/EinsumEinsum9transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_1/self_attention/key/einsum/Einsumł 9transformer/layer_1/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_1_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_1/self_attention/key/add/ReadVariableOp¦ *transformer/layer_1/self_attention/key/addAddV2=transformer/layer_1/self_attention/key/einsum/Einsum:output:0Atransformer/layer_1/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_1/self_attention/key/add¢ Etransformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_1/self_attention/value/einsum/EinsumEinsum9transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_1/self_attention/value/einsum/Einsum’ ;transformer/layer_1/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_1_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_1/self_attention/value/add/ReadVariableOp® ,transformer/layer_1/self_attention/value/addAddV2?transformer/layer_1/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_1/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_1/self_attention/value/addÄ 0transformer/layer_1/self_attention/einsum/EinsumEinsum.transformer/layer_1/self_attention/key/add:z:00transformer/layer_1/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_1/self_attention/einsum/Einsum™ (transformer/layer_1/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_1/self_attention/Mul/y‘ &transformer/layer_1/self_attention/MulMul9transformer/layer_1/self_attention/einsum/Einsum:output:01transformer/layer_1/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_1/self_attention/MulŅ Btransformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dimÄ >transformer/layer_1/self_attention/masked_softmax_1/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_1/self_attention/masked_softmax_1/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_1/self_attention/masked_softmax_1/ExpandDims» 9transformer/layer_1/self_attention/masked_softmax_1/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_1/self_attention/masked_softmax_1/sub/xŅ 7transformer/layer_1/self_attention/masked_softmax_1/subSubBtransformer/layer_1/self_attention/masked_softmax_1/sub/x:output:0Gtransformer/layer_1/self_attention/masked_softmax_1/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_1/self_attention/masked_softmax_1/sub» 9transformer/layer_1/self_attention/masked_softmax_1/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_1/self_attention/masked_softmax_1/mul/yĘ 7transformer/layer_1/self_attention/masked_softmax_1/mulMul;transformer/layer_1/self_attention/masked_softmax_1/sub:z:0Btransformer/layer_1/self_attention/masked_softmax_1/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_1/self_attention/masked_softmax_1/mul° 7transformer/layer_1/self_attention/masked_softmax_1/addAddV2*transformer/layer_1/self_attention/Mul:z:0;transformer/layer_1/self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_1/self_attention/masked_softmax_1/addŽ ;transformer/layer_1/self_attention/masked_softmax_1/SoftmaxSoftmax;transformer/layer_1/self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_1/self_attention/masked_softmax_1/Softmax½ :transformer/layer_1/self_attention/dropout_3/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2< :transformer/layer_1/self_attention/dropout_3/dropout/ConstÓ 8transformer/layer_1/self_attention/dropout_3/dropout/MulMulEtransformer/layer_1/self_attention/masked_softmax_1/Softmax:softmax:0Ctransformer/layer_1/self_attention/dropout_3/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2: 8transformer/layer_1/self_attention/dropout_3/dropout/Mulķ :transformer/layer_1/self_attention/dropout_3/dropout/ShapeShapeEtransformer/layer_1/self_attention/masked_softmax_1/Softmax:softmax:0* T0* _output_shapes :2< :transformer/layer_1/self_attention/dropout_3/dropout/ShapeÕ Qtransformer/layer_1/self_attention/dropout_3/dropout/random_uniform/RandomUniform RandomUniformCtransformer/layer_1/self_attention/dropout_3/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02S Qtransformer/layer_1/self_attention/dropout_3/dropout/random_uniform/RandomUniformĻ Ctransformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2E Ctransformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual/yŒ Atransformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual GreaterEqualZtransformer/layer_1/self_attention/dropout_3/dropout/random_uniform/RandomUniform:output:0Ltransformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2C Atransformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual  9transformer/layer_1/self_attention/dropout_3/dropout/CastCastEtransformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_1/self_attention/dropout_3/dropout/CastČ :transformer/layer_1/self_attention/dropout_3/dropout/Mul_1Multransformer/layer_1/self_attention/dropout_3/dropout/Mul_1:z:00transformer/layer_1/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_1/self_attention/einsum_1/EinsumĆ Ptransformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_1/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_1/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_1/self_attention/attention_output/einsum/Einsum Ftransformer/layer_1/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_1_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_1/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_1/self_attention/attention_output/addAddV2Jtransformer/layer_1/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_1/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_1/self_attention/attention_output/addŸ +transformer/layer_1/dropout_4/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2- +transformer/layer_1/dropout_4/dropout/Const )transformer/layer_1/dropout_4/dropout/MulMul;transformer/layer_1/self_attention/attention_output/add:z:04transformer/layer_1/dropout_4/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer/layer_1/dropout_4/dropout/MulÅ +transformer/layer_1/dropout_4/dropout/ShapeShape;transformer/layer_1/self_attention/attention_output/add:z:0* T0* _output_shapes :2- +transformer/layer_1/dropout_4/dropout/Shapeœ Btransformer/layer_1/dropout_4/dropout/random_uniform/RandomUniform RandomUniform4transformer/layer_1/dropout_4/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02D Btransformer/layer_1/dropout_4/dropout/random_uniform/RandomUniform± 4transformer/layer_1/dropout_4/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=26 4transformer/layer_1/dropout_4/dropout/GreaterEqual/yÄ 2transformer/layer_1/dropout_4/dropout/GreaterEqual GreaterEqualKtransformer/layer_1/dropout_4/dropout/random_uniform/RandomUniform:output:0=transformer/layer_1/dropout_4/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer/layer_1/dropout_4/dropout/GreaterEqualē *transformer/layer_1/dropout_4/dropout/CastCast6transformer/layer_1/dropout_4/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_1/dropout_4/dropout/Cast€ +transformer/layer_1/dropout_4/dropout/Mul_1Mul-transformer/layer_1/dropout_4/dropout/Mul:z:0.transformer/layer_1/dropout_4/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_1/dropout_4/dropout/Mul_1ē transformer/layer_1/addAddV29transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0/transformer/layer_1/dropout_4/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_1/addę Ltransformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_1/self_attention_layer_norm/moments/meanMeantransformer/layer_1/add:z:0Utransformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_1/self_attention_layer_norm/moments/meanœ Btransformer/layer_1/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_1/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_1/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_1/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_1/add:z:0Ktransformer/layer_1/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_1/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_1/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_1/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_1/self_attention_layer_norm/moments/varianceĆ =transformer/layer_1/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_1/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_1/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_1/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_1/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_1/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_1/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_1/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_1/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_1/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_1/add:z:0?transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_1/self_attention_layer_norm/moments/mean:output:0?transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_1/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_1/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_1/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_1/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_1/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_1/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_1/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_1_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_1/intermediate/einsum/EinsumEinsumAtransformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_1/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_1/intermediate/einsum/Einsumä 3transformer/layer_1/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_1/output/einsum/EinsumŅ -transformer/layer_1/output/add/ReadVariableOpReadVariableOp6transformer_layer_1_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_1/output/add/ReadVariableOpó transformer/layer_1/output/addAddV21transformer/layer_1/output/einsum/Einsum:output:05transformer/layer_1/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_1/output/addŸ +transformer/layer_1/dropout_5/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2- +transformer/layer_1/dropout_5/dropout/Const÷ )transformer/layer_1/dropout_5/dropout/MulMul"transformer/layer_1/output/add:z:04transformer/layer_1/dropout_5/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer/layer_1/dropout_5/dropout/Mul¬ +transformer/layer_1/dropout_5/dropout/ShapeShape"transformer/layer_1/output/add:z:0* T0* _output_shapes :2- +transformer/layer_1/dropout_5/dropout/Shapeœ Btransformer/layer_1/dropout_5/dropout/random_uniform/RandomUniform RandomUniform4transformer/layer_1/dropout_5/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02D Btransformer/layer_1/dropout_5/dropout/random_uniform/RandomUniform± 4transformer/layer_1/dropout_5/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=26 4transformer/layer_1/dropout_5/dropout/GreaterEqual/yÄ 2transformer/layer_1/dropout_5/dropout/GreaterEqual GreaterEqualKtransformer/layer_1/dropout_5/dropout/random_uniform/RandomUniform:output:0=transformer/layer_1/dropout_5/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer/layer_1/dropout_5/dropout/GreaterEqualē *transformer/layer_1/dropout_5/dropout/CastCast6transformer/layer_1/dropout_5/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_1/dropout_5/dropout/Cast€ +transformer/layer_1/dropout_5/dropout/Mul_1Mul-transformer/layer_1/dropout_5/dropout/Mul:z:0.transformer/layer_1/dropout_5/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_1/dropout_5/dropout/Mul_1ó transformer/layer_1/add_1AddV2/transformer/layer_1/dropout_5/dropout/Mul_1:z:0Atransformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_1/add_1Ö Dtransformer/layer_1/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_1/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_1/output_layer_norm/moments/meanMeantransformer/layer_1/add_1:z:0Mtransformer/layer_1/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_1/output_layer_norm/moments/mean„ :transformer/layer_1/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_1/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_1/output_layer_norm/moments/StopGradient» ?transformer/layer_1/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_1/add_1:z:0Ctransformer/layer_1/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_1/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_1/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_1/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_1/output_layer_norm/moments/varianceMeanCtransformer/layer_1/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_1/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_1/output_layer_norm/moments/variance³ 5transformer/layer_1/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_1/output_layer_norm/batchnorm/add/y³ 3transformer/layer_1/output_layer_norm/batchnorm/addAddV2?transformer/layer_1/output_layer_norm/moments/variance:output:0>transformer/layer_1/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_1/output_layer_norm/batchnorm/addļ 5transformer/layer_1/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_1/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_1/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_1/output_layer_norm/batchnorm/mulMul9transformer/layer_1/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_1/output_layer_norm/batchnorm/mul 5transformer/layer_1/output_layer_norm/batchnorm/mul_1Multransformer/layer_1/add_1:z:07transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_1/output_layer_norm/batchnorm/mul_1« 5transformer/layer_1/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_1/output_layer_norm/moments/mean:output:07transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_1/output_layer_norm/batchnorm/mul_2… >transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_1_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_1/output_layer_norm/batchnorm/subSubFtransformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_1/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_1/output_layer_norm/batchnorm/sub« 5transformer/layer_1/output_layer_norm/batchnorm/add_1AddV29transformer/layer_1/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_1/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_1/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_2/self_attention/query/einsum/EinsumEinsum9transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_2/self_attention/query/einsum/Einsum’ ;transformer/layer_2/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_2_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_2/self_attention/query/add/ReadVariableOp® ,transformer/layer_2/self_attention/query/addAddV2?transformer/layer_2/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_2/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_2/self_attention/query/addœ Ctransformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_2/self_attention/key/einsum/EinsumEinsum9transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_2/self_attention/key/einsum/Einsumł 9transformer/layer_2/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_2_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_2/self_attention/key/add/ReadVariableOp¦ *transformer/layer_2/self_attention/key/addAddV2=transformer/layer_2/self_attention/key/einsum/Einsum:output:0Atransformer/layer_2/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_2/self_attention/key/add¢ Etransformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_2/self_attention/value/einsum/EinsumEinsum9transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_2/self_attention/value/einsum/Einsum’ ;transformer/layer_2/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_2_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_2/self_attention/value/add/ReadVariableOp® ,transformer/layer_2/self_attention/value/addAddV2?transformer/layer_2/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_2/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_2/self_attention/value/addÄ 0transformer/layer_2/self_attention/einsum/EinsumEinsum.transformer/layer_2/self_attention/key/add:z:00transformer/layer_2/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_2/self_attention/einsum/Einsum™ (transformer/layer_2/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_2/self_attention/Mul/y‘ &transformer/layer_2/self_attention/MulMul9transformer/layer_2/self_attention/einsum/Einsum:output:01transformer/layer_2/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_2/self_attention/MulŅ Btransformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dimÄ >transformer/layer_2/self_attention/masked_softmax_2/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_2/self_attention/masked_softmax_2/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_2/self_attention/masked_softmax_2/ExpandDims» 9transformer/layer_2/self_attention/masked_softmax_2/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_2/self_attention/masked_softmax_2/sub/xŅ 7transformer/layer_2/self_attention/masked_softmax_2/subSubBtransformer/layer_2/self_attention/masked_softmax_2/sub/x:output:0Gtransformer/layer_2/self_attention/masked_softmax_2/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_2/self_attention/masked_softmax_2/sub» 9transformer/layer_2/self_attention/masked_softmax_2/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_2/self_attention/masked_softmax_2/mul/yĘ 7transformer/layer_2/self_attention/masked_softmax_2/mulMul;transformer/layer_2/self_attention/masked_softmax_2/sub:z:0Btransformer/layer_2/self_attention/masked_softmax_2/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_2/self_attention/masked_softmax_2/mul° 7transformer/layer_2/self_attention/masked_softmax_2/addAddV2*transformer/layer_2/self_attention/Mul:z:0;transformer/layer_2/self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_2/self_attention/masked_softmax_2/addŽ ;transformer/layer_2/self_attention/masked_softmax_2/SoftmaxSoftmax;transformer/layer_2/self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_2/self_attention/masked_softmax_2/Softmax½ :transformer/layer_2/self_attention/dropout_6/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2< :transformer/layer_2/self_attention/dropout_6/dropout/ConstÓ 8transformer/layer_2/self_attention/dropout_6/dropout/MulMulEtransformer/layer_2/self_attention/masked_softmax_2/Softmax:softmax:0Ctransformer/layer_2/self_attention/dropout_6/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2: 8transformer/layer_2/self_attention/dropout_6/dropout/Mulķ :transformer/layer_2/self_attention/dropout_6/dropout/ShapeShapeEtransformer/layer_2/self_attention/masked_softmax_2/Softmax:softmax:0* T0* _output_shapes :2< :transformer/layer_2/self_attention/dropout_6/dropout/ShapeÕ Qtransformer/layer_2/self_attention/dropout_6/dropout/random_uniform/RandomUniform RandomUniformCtransformer/layer_2/self_attention/dropout_6/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02S Qtransformer/layer_2/self_attention/dropout_6/dropout/random_uniform/RandomUniformĻ Ctransformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2E Ctransformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual/yŒ Atransformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual GreaterEqualZtransformer/layer_2/self_attention/dropout_6/dropout/random_uniform/RandomUniform:output:0Ltransformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2C Atransformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual  9transformer/layer_2/self_attention/dropout_6/dropout/CastCastEtransformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_2/self_attention/dropout_6/dropout/CastČ :transformer/layer_2/self_attention/dropout_6/dropout/Mul_1Multransformer/layer_2/self_attention/dropout_6/dropout/Mul_1:z:00transformer/layer_2/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_2/self_attention/einsum_1/EinsumĆ Ptransformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_2/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_2/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_2/self_attention/attention_output/einsum/Einsum Ftransformer/layer_2/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_2_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_2/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_2/self_attention/attention_output/addAddV2Jtransformer/layer_2/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_2/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_2/self_attention/attention_output/addŸ +transformer/layer_2/dropout_7/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2- +transformer/layer_2/dropout_7/dropout/Const )transformer/layer_2/dropout_7/dropout/MulMul;transformer/layer_2/self_attention/attention_output/add:z:04transformer/layer_2/dropout_7/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer/layer_2/dropout_7/dropout/MulÅ +transformer/layer_2/dropout_7/dropout/ShapeShape;transformer/layer_2/self_attention/attention_output/add:z:0* T0* _output_shapes :2- +transformer/layer_2/dropout_7/dropout/Shapeœ Btransformer/layer_2/dropout_7/dropout/random_uniform/RandomUniform RandomUniform4transformer/layer_2/dropout_7/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02D Btransformer/layer_2/dropout_7/dropout/random_uniform/RandomUniform± 4transformer/layer_2/dropout_7/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=26 4transformer/layer_2/dropout_7/dropout/GreaterEqual/yÄ 2transformer/layer_2/dropout_7/dropout/GreaterEqual GreaterEqualKtransformer/layer_2/dropout_7/dropout/random_uniform/RandomUniform:output:0=transformer/layer_2/dropout_7/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer/layer_2/dropout_7/dropout/GreaterEqualē *transformer/layer_2/dropout_7/dropout/CastCast6transformer/layer_2/dropout_7/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_2/dropout_7/dropout/Cast€ +transformer/layer_2/dropout_7/dropout/Mul_1Mul-transformer/layer_2/dropout_7/dropout/Mul:z:0.transformer/layer_2/dropout_7/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_2/dropout_7/dropout/Mul_1ē transformer/layer_2/addAddV29transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0/transformer/layer_2/dropout_7/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_2/addę Ltransformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_2/self_attention_layer_norm/moments/meanMeantransformer/layer_2/add:z:0Utransformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_2/self_attention_layer_norm/moments/meanœ Btransformer/layer_2/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_2/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_2/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_2/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_2/add:z:0Ktransformer/layer_2/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_2/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_2/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_2/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_2/self_attention_layer_norm/moments/varianceĆ =transformer/layer_2/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_2/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_2/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_2/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_2/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_2/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_2/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_2/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_2/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_2/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_2/add:z:0?transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_2/self_attention_layer_norm/moments/mean:output:0?transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_2/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_2/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_2/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_2/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_2/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_2/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_2/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_2_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_2/intermediate/einsum/EinsumEinsumAtransformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_2/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_2/intermediate/einsum/Einsumä 3transformer/layer_2/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_2/output/einsum/EinsumŅ -transformer/layer_2/output/add/ReadVariableOpReadVariableOp6transformer_layer_2_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_2/output/add/ReadVariableOpó transformer/layer_2/output/addAddV21transformer/layer_2/output/einsum/Einsum:output:05transformer/layer_2/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_2/output/addŸ +transformer/layer_2/dropout_8/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2- +transformer/layer_2/dropout_8/dropout/Const÷ )transformer/layer_2/dropout_8/dropout/MulMul"transformer/layer_2/output/add:z:04transformer/layer_2/dropout_8/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer/layer_2/dropout_8/dropout/Mul¬ +transformer/layer_2/dropout_8/dropout/ShapeShape"transformer/layer_2/output/add:z:0* T0* _output_shapes :2- +transformer/layer_2/dropout_8/dropout/Shapeœ Btransformer/layer_2/dropout_8/dropout/random_uniform/RandomUniform RandomUniform4transformer/layer_2/dropout_8/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02D Btransformer/layer_2/dropout_8/dropout/random_uniform/RandomUniform± 4transformer/layer_2/dropout_8/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=26 4transformer/layer_2/dropout_8/dropout/GreaterEqual/yÄ 2transformer/layer_2/dropout_8/dropout/GreaterEqual GreaterEqualKtransformer/layer_2/dropout_8/dropout/random_uniform/RandomUniform:output:0=transformer/layer_2/dropout_8/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer/layer_2/dropout_8/dropout/GreaterEqualē *transformer/layer_2/dropout_8/dropout/CastCast6transformer/layer_2/dropout_8/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_2/dropout_8/dropout/Cast€ +transformer/layer_2/dropout_8/dropout/Mul_1Mul-transformer/layer_2/dropout_8/dropout/Mul:z:0.transformer/layer_2/dropout_8/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_2/dropout_8/dropout/Mul_1ó transformer/layer_2/add_1AddV2/transformer/layer_2/dropout_8/dropout/Mul_1:z:0Atransformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_2/add_1Ö Dtransformer/layer_2/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_2/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_2/output_layer_norm/moments/meanMeantransformer/layer_2/add_1:z:0Mtransformer/layer_2/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_2/output_layer_norm/moments/mean„ :transformer/layer_2/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_2/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_2/output_layer_norm/moments/StopGradient» ?transformer/layer_2/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_2/add_1:z:0Ctransformer/layer_2/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_2/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_2/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_2/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_2/output_layer_norm/moments/varianceMeanCtransformer/layer_2/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_2/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_2/output_layer_norm/moments/variance³ 5transformer/layer_2/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_2/output_layer_norm/batchnorm/add/y³ 3transformer/layer_2/output_layer_norm/batchnorm/addAddV2?transformer/layer_2/output_layer_norm/moments/variance:output:0>transformer/layer_2/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_2/output_layer_norm/batchnorm/addļ 5transformer/layer_2/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_2/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_2/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_2/output_layer_norm/batchnorm/mulMul9transformer/layer_2/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_2/output_layer_norm/batchnorm/mul 5transformer/layer_2/output_layer_norm/batchnorm/mul_1Multransformer/layer_2/add_1:z:07transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_2/output_layer_norm/batchnorm/mul_1« 5transformer/layer_2/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_2/output_layer_norm/moments/mean:output:07transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_2/output_layer_norm/batchnorm/mul_2… >transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_2_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_2/output_layer_norm/batchnorm/subSubFtransformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_2/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_2/output_layer_norm/batchnorm/sub« 5transformer/layer_2/output_layer_norm/batchnorm/add_1AddV29transformer/layer_2/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_2/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_2/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_3/self_attention/query/einsum/EinsumEinsum9transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_3/self_attention/query/einsum/Einsum’ ;transformer/layer_3/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_3_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_3/self_attention/query/add/ReadVariableOp® ,transformer/layer_3/self_attention/query/addAddV2?transformer/layer_3/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_3/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_3/self_attention/query/addœ Ctransformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_3/self_attention/key/einsum/EinsumEinsum9transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_3/self_attention/key/einsum/Einsumł 9transformer/layer_3/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_3_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_3/self_attention/key/add/ReadVariableOp¦ *transformer/layer_3/self_attention/key/addAddV2=transformer/layer_3/self_attention/key/einsum/Einsum:output:0Atransformer/layer_3/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_3/self_attention/key/add¢ Etransformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_3/self_attention/value/einsum/EinsumEinsum9transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_3/self_attention/value/einsum/Einsum’ ;transformer/layer_3/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_3_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_3/self_attention/value/add/ReadVariableOp® ,transformer/layer_3/self_attention/value/addAddV2?transformer/layer_3/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_3/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_3/self_attention/value/addÄ 0transformer/layer_3/self_attention/einsum/EinsumEinsum.transformer/layer_3/self_attention/key/add:z:00transformer/layer_3/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_3/self_attention/einsum/Einsum™ (transformer/layer_3/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_3/self_attention/Mul/y‘ &transformer/layer_3/self_attention/MulMul9transformer/layer_3/self_attention/einsum/Einsum:output:01transformer/layer_3/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_3/self_attention/MulŅ Btransformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dimÄ >transformer/layer_3/self_attention/masked_softmax_3/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_3/self_attention/masked_softmax_3/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_3/self_attention/masked_softmax_3/ExpandDims» 9transformer/layer_3/self_attention/masked_softmax_3/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_3/self_attention/masked_softmax_3/sub/xŅ 7transformer/layer_3/self_attention/masked_softmax_3/subSubBtransformer/layer_3/self_attention/masked_softmax_3/sub/x:output:0Gtransformer/layer_3/self_attention/masked_softmax_3/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_3/self_attention/masked_softmax_3/sub» 9transformer/layer_3/self_attention/masked_softmax_3/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_3/self_attention/masked_softmax_3/mul/yĘ 7transformer/layer_3/self_attention/masked_softmax_3/mulMul;transformer/layer_3/self_attention/masked_softmax_3/sub:z:0Btransformer/layer_3/self_attention/masked_softmax_3/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_3/self_attention/masked_softmax_3/mul° 7transformer/layer_3/self_attention/masked_softmax_3/addAddV2*transformer/layer_3/self_attention/Mul:z:0;transformer/layer_3/self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_3/self_attention/masked_softmax_3/addŽ ;transformer/layer_3/self_attention/masked_softmax_3/SoftmaxSoftmax;transformer/layer_3/self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_3/self_attention/masked_softmax_3/Softmax½ :transformer/layer_3/self_attention/dropout_9/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2< :transformer/layer_3/self_attention/dropout_9/dropout/ConstÓ 8transformer/layer_3/self_attention/dropout_9/dropout/MulMulEtransformer/layer_3/self_attention/masked_softmax_3/Softmax:softmax:0Ctransformer/layer_3/self_attention/dropout_9/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2: 8transformer/layer_3/self_attention/dropout_9/dropout/Mulķ :transformer/layer_3/self_attention/dropout_9/dropout/ShapeShapeEtransformer/layer_3/self_attention/masked_softmax_3/Softmax:softmax:0* T0* _output_shapes :2< :transformer/layer_3/self_attention/dropout_9/dropout/ShapeÕ Qtransformer/layer_3/self_attention/dropout_9/dropout/random_uniform/RandomUniform RandomUniformCtransformer/layer_3/self_attention/dropout_9/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02S Qtransformer/layer_3/self_attention/dropout_9/dropout/random_uniform/RandomUniformĻ Ctransformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2E Ctransformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual/yŒ Atransformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual GreaterEqualZtransformer/layer_3/self_attention/dropout_9/dropout/random_uniform/RandomUniform:output:0Ltransformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2C Atransformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual  9transformer/layer_3/self_attention/dropout_9/dropout/CastCastEtransformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_3/self_attention/dropout_9/dropout/CastČ :transformer/layer_3/self_attention/dropout_9/dropout/Mul_1Multransformer/layer_3/self_attention/dropout_9/dropout/Mul_1:z:00transformer/layer_3/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_3/self_attention/einsum_1/EinsumĆ Ptransformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_3/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_3/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_3/self_attention/attention_output/einsum/Einsum Ftransformer/layer_3/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_3_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_3/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_3/self_attention/attention_output/addAddV2Jtransformer/layer_3/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_3/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_3/self_attention/attention_output/add” ,transformer/layer_3/dropout_10/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_3/dropout_10/dropout/Const“ *transformer/layer_3/dropout_10/dropout/MulMul;transformer/layer_3/self_attention/attention_output/add:z:05transformer/layer_3/dropout_10/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_3/dropout_10/dropout/MulĒ ,transformer/layer_3/dropout_10/dropout/ShapeShape;transformer/layer_3/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_3/dropout_10/dropout/ShapeŸ Ctransformer/layer_3/dropout_10/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_3/dropout_10/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_3/dropout_10/dropout/random_uniform/RandomUniform³ 5transformer/layer_3/dropout_10/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_3/dropout_10/dropout/GreaterEqual/yČ 3transformer/layer_3/dropout_10/dropout/GreaterEqual GreaterEqualLtransformer/layer_3/dropout_10/dropout/random_uniform/RandomUniform:output:0>transformer/layer_3/dropout_10/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_3/dropout_10/dropout/GreaterEqualź +transformer/layer_3/dropout_10/dropout/CastCast7transformer/layer_3/dropout_10/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_3/dropout_10/dropout/Cast„ ,transformer/layer_3/dropout_10/dropout/Mul_1Mul.transformer/layer_3/dropout_10/dropout/Mul:z:0/transformer/layer_3/dropout_10/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_3/dropout_10/dropout/Mul_1č transformer/layer_3/addAddV29transformer/layer_2/output_layer_norm/batchnorm/add_1:z:00transformer/layer_3/dropout_10/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_3/addę Ltransformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_3/self_attention_layer_norm/moments/meanMeantransformer/layer_3/add:z:0Utransformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_3/self_attention_layer_norm/moments/meanœ Btransformer/layer_3/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_3/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_3/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_3/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_3/add:z:0Ktransformer/layer_3/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_3/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_3/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_3/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_3/self_attention_layer_norm/moments/varianceĆ =transformer/layer_3/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_3/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_3/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_3/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_3/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_3/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_3/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_3/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_3/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_3/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_3/add:z:0?transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_3/self_attention_layer_norm/moments/mean:output:0?transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_3/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_3/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_3/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_3/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_3/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_3/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_3/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_3_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_3/intermediate/einsum/EinsumEinsumAtransformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_3/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_3/intermediate/einsum/Einsumä 3transformer/layer_3/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_3/output/einsum/EinsumŅ -transformer/layer_3/output/add/ReadVariableOpReadVariableOp6transformer_layer_3_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_3/output/add/ReadVariableOpó transformer/layer_3/output/addAddV21transformer/layer_3/output/einsum/Einsum:output:05transformer/layer_3/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_3/output/add” ,transformer/layer_3/dropout_11/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_3/dropout_11/dropout/Constś *transformer/layer_3/dropout_11/dropout/MulMul"transformer/layer_3/output/add:z:05transformer/layer_3/dropout_11/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_3/dropout_11/dropout/Mul® ,transformer/layer_3/dropout_11/dropout/ShapeShape"transformer/layer_3/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_3/dropout_11/dropout/ShapeŸ Ctransformer/layer_3/dropout_11/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_3/dropout_11/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_3/dropout_11/dropout/random_uniform/RandomUniform³ 5transformer/layer_3/dropout_11/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_3/dropout_11/dropout/GreaterEqual/yČ 3transformer/layer_3/dropout_11/dropout/GreaterEqual GreaterEqualLtransformer/layer_3/dropout_11/dropout/random_uniform/RandomUniform:output:0>transformer/layer_3/dropout_11/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_3/dropout_11/dropout/GreaterEqualź +transformer/layer_3/dropout_11/dropout/CastCast7transformer/layer_3/dropout_11/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_3/dropout_11/dropout/Cast„ ,transformer/layer_3/dropout_11/dropout/Mul_1Mul.transformer/layer_3/dropout_11/dropout/Mul:z:0/transformer/layer_3/dropout_11/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_3/dropout_11/dropout/Mul_1ō transformer/layer_3/add_1AddV20transformer/layer_3/dropout_11/dropout/Mul_1:z:0Atransformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_3/add_1Ö Dtransformer/layer_3/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_3/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_3/output_layer_norm/moments/meanMeantransformer/layer_3/add_1:z:0Mtransformer/layer_3/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_3/output_layer_norm/moments/mean„ :transformer/layer_3/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_3/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_3/output_layer_norm/moments/StopGradient» ?transformer/layer_3/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_3/add_1:z:0Ctransformer/layer_3/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_3/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_3/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_3/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_3/output_layer_norm/moments/varianceMeanCtransformer/layer_3/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_3/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_3/output_layer_norm/moments/variance³ 5transformer/layer_3/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_3/output_layer_norm/batchnorm/add/y³ 3transformer/layer_3/output_layer_norm/batchnorm/addAddV2?transformer/layer_3/output_layer_norm/moments/variance:output:0>transformer/layer_3/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_3/output_layer_norm/batchnorm/addļ 5transformer/layer_3/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_3/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_3/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_3/output_layer_norm/batchnorm/mulMul9transformer/layer_3/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_3/output_layer_norm/batchnorm/mul 5transformer/layer_3/output_layer_norm/batchnorm/mul_1Multransformer/layer_3/add_1:z:07transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_3/output_layer_norm/batchnorm/mul_1« 5transformer/layer_3/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_3/output_layer_norm/moments/mean:output:07transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_3/output_layer_norm/batchnorm/mul_2… >transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_3_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_3/output_layer_norm/batchnorm/subSubFtransformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_3/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_3/output_layer_norm/batchnorm/sub« 5transformer/layer_3/output_layer_norm/batchnorm/add_1AddV29transformer/layer_3/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_3/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_3/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_4/self_attention/query/einsum/EinsumEinsum9transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_4/self_attention/query/einsum/Einsum’ ;transformer/layer_4/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_4_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_4/self_attention/query/add/ReadVariableOp® ,transformer/layer_4/self_attention/query/addAddV2?transformer/layer_4/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_4/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_4/self_attention/query/addœ Ctransformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_4/self_attention/key/einsum/EinsumEinsum9transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_4/self_attention/key/einsum/Einsumł 9transformer/layer_4/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_4_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_4/self_attention/key/add/ReadVariableOp¦ *transformer/layer_4/self_attention/key/addAddV2=transformer/layer_4/self_attention/key/einsum/Einsum:output:0Atransformer/layer_4/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_4/self_attention/key/add¢ Etransformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_4/self_attention/value/einsum/EinsumEinsum9transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_4/self_attention/value/einsum/Einsum’ ;transformer/layer_4/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_4_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_4/self_attention/value/add/ReadVariableOp® ,transformer/layer_4/self_attention/value/addAddV2?transformer/layer_4/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_4/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_4/self_attention/value/addÄ 0transformer/layer_4/self_attention/einsum/EinsumEinsum.transformer/layer_4/self_attention/key/add:z:00transformer/layer_4/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_4/self_attention/einsum/Einsum™ (transformer/layer_4/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_4/self_attention/Mul/y‘ &transformer/layer_4/self_attention/MulMul9transformer/layer_4/self_attention/einsum/Einsum:output:01transformer/layer_4/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_4/self_attention/MulŅ Btransformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dimÄ >transformer/layer_4/self_attention/masked_softmax_4/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_4/self_attention/masked_softmax_4/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_4/self_attention/masked_softmax_4/ExpandDims» 9transformer/layer_4/self_attention/masked_softmax_4/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_4/self_attention/masked_softmax_4/sub/xŅ 7transformer/layer_4/self_attention/masked_softmax_4/subSubBtransformer/layer_4/self_attention/masked_softmax_4/sub/x:output:0Gtransformer/layer_4/self_attention/masked_softmax_4/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_4/self_attention/masked_softmax_4/sub» 9transformer/layer_4/self_attention/masked_softmax_4/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_4/self_attention/masked_softmax_4/mul/yĘ 7transformer/layer_4/self_attention/masked_softmax_4/mulMul;transformer/layer_4/self_attention/masked_softmax_4/sub:z:0Btransformer/layer_4/self_attention/masked_softmax_4/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_4/self_attention/masked_softmax_4/mul° 7transformer/layer_4/self_attention/masked_softmax_4/addAddV2*transformer/layer_4/self_attention/Mul:z:0;transformer/layer_4/self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_4/self_attention/masked_softmax_4/addŽ ;transformer/layer_4/self_attention/masked_softmax_4/SoftmaxSoftmax;transformer/layer_4/self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_4/self_attention/masked_softmax_4/Softmaxæ ;transformer/layer_4/self_attention/dropout_12/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2= ;transformer/layer_4/self_attention/dropout_12/dropout/ConstÖ 9transformer/layer_4/self_attention/dropout_12/dropout/MulMulEtransformer/layer_4/self_attention/masked_softmax_4/Softmax:softmax:0Dtransformer/layer_4/self_attention/dropout_12/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_4/self_attention/dropout_12/dropout/Mulļ ;transformer/layer_4/self_attention/dropout_12/dropout/ShapeShapeEtransformer/layer_4/self_attention/masked_softmax_4/Softmax:softmax:0* T0* _output_shapes :2= ;transformer/layer_4/self_attention/dropout_12/dropout/ShapeŲ Rtransformer/layer_4/self_attention/dropout_12/dropout/random_uniform/RandomUniform RandomUniformDtransformer/layer_4/self_attention/dropout_12/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02T Rtransformer/layer_4/self_attention/dropout_12/dropout/random_uniform/RandomUniformŃ Dtransformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2F Dtransformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual/y Btransformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual GreaterEqual[transformer/layer_4/self_attention/dropout_12/dropout/random_uniform/RandomUniform:output:0Mtransformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual£ :transformer/layer_4/self_attention/dropout_12/dropout/CastCastFtransformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_4/self_attention/dropout_12/dropout/CastĢ ;transformer/layer_4/self_attention/dropout_12/dropout/Mul_1Mul=transformer/layer_4/self_attention/dropout_12/dropout/Mul:z:0>transformer/layer_4/self_attention/dropout_12/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_4/self_attention/dropout_12/dropout/Mul_1Š 2transformer/layer_4/self_attention/einsum_1/EinsumEinsum?transformer/layer_4/self_attention/dropout_12/dropout/Mul_1:z:00transformer/layer_4/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_4/self_attention/einsum_1/EinsumĆ Ptransformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_4/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_4/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_4/self_attention/attention_output/einsum/Einsum Ftransformer/layer_4/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_4_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_4/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_4/self_attention/attention_output/addAddV2Jtransformer/layer_4/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_4/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_4/self_attention/attention_output/add” ,transformer/layer_4/dropout_13/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_4/dropout_13/dropout/Const“ *transformer/layer_4/dropout_13/dropout/MulMul;transformer/layer_4/self_attention/attention_output/add:z:05transformer/layer_4/dropout_13/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_4/dropout_13/dropout/MulĒ ,transformer/layer_4/dropout_13/dropout/ShapeShape;transformer/layer_4/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_4/dropout_13/dropout/ShapeŸ Ctransformer/layer_4/dropout_13/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_4/dropout_13/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_4/dropout_13/dropout/random_uniform/RandomUniform³ 5transformer/layer_4/dropout_13/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_4/dropout_13/dropout/GreaterEqual/yČ 3transformer/layer_4/dropout_13/dropout/GreaterEqual GreaterEqualLtransformer/layer_4/dropout_13/dropout/random_uniform/RandomUniform:output:0>transformer/layer_4/dropout_13/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_4/dropout_13/dropout/GreaterEqualź +transformer/layer_4/dropout_13/dropout/CastCast7transformer/layer_4/dropout_13/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_4/dropout_13/dropout/Cast„ ,transformer/layer_4/dropout_13/dropout/Mul_1Mul.transformer/layer_4/dropout_13/dropout/Mul:z:0/transformer/layer_4/dropout_13/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_4/dropout_13/dropout/Mul_1č transformer/layer_4/addAddV29transformer/layer_3/output_layer_norm/batchnorm/add_1:z:00transformer/layer_4/dropout_13/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_4/addę Ltransformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_4/self_attention_layer_norm/moments/meanMeantransformer/layer_4/add:z:0Utransformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_4/self_attention_layer_norm/moments/meanœ Btransformer/layer_4/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_4/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_4/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_4/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_4/add:z:0Ktransformer/layer_4/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_4/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_4/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_4/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_4/self_attention_layer_norm/moments/varianceĆ =transformer/layer_4/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_4/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_4/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_4/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_4/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_4/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_4/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_4/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_4/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_4/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_4/add:z:0?transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_4/self_attention_layer_norm/moments/mean:output:0?transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_4/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_4/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_4/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_4/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_4/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_4/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_4/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_4_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_4/intermediate/einsum/EinsumEinsumAtransformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_4/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_4/intermediate/einsum/Einsumä 3transformer/layer_4/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_4/output/einsum/EinsumŅ -transformer/layer_4/output/add/ReadVariableOpReadVariableOp6transformer_layer_4_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_4/output/add/ReadVariableOpó transformer/layer_4/output/addAddV21transformer/layer_4/output/einsum/Einsum:output:05transformer/layer_4/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_4/output/add” ,transformer/layer_4/dropout_14/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_4/dropout_14/dropout/Constś *transformer/layer_4/dropout_14/dropout/MulMul"transformer/layer_4/output/add:z:05transformer/layer_4/dropout_14/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_4/dropout_14/dropout/Mul® ,transformer/layer_4/dropout_14/dropout/ShapeShape"transformer/layer_4/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_4/dropout_14/dropout/ShapeŸ Ctransformer/layer_4/dropout_14/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_4/dropout_14/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_4/dropout_14/dropout/random_uniform/RandomUniform³ 5transformer/layer_4/dropout_14/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_4/dropout_14/dropout/GreaterEqual/yČ 3transformer/layer_4/dropout_14/dropout/GreaterEqual GreaterEqualLtransformer/layer_4/dropout_14/dropout/random_uniform/RandomUniform:output:0>transformer/layer_4/dropout_14/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_4/dropout_14/dropout/GreaterEqualź +transformer/layer_4/dropout_14/dropout/CastCast7transformer/layer_4/dropout_14/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_4/dropout_14/dropout/Cast„ ,transformer/layer_4/dropout_14/dropout/Mul_1Mul.transformer/layer_4/dropout_14/dropout/Mul:z:0/transformer/layer_4/dropout_14/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_4/dropout_14/dropout/Mul_1ō transformer/layer_4/add_1AddV20transformer/layer_4/dropout_14/dropout/Mul_1:z:0Atransformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_4/add_1Ö Dtransformer/layer_4/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_4/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_4/output_layer_norm/moments/meanMeantransformer/layer_4/add_1:z:0Mtransformer/layer_4/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_4/output_layer_norm/moments/mean„ :transformer/layer_4/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_4/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_4/output_layer_norm/moments/StopGradient» ?transformer/layer_4/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_4/add_1:z:0Ctransformer/layer_4/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_4/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_4/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_4/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_4/output_layer_norm/moments/varianceMeanCtransformer/layer_4/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_4/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_4/output_layer_norm/moments/variance³ 5transformer/layer_4/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_4/output_layer_norm/batchnorm/add/y³ 3transformer/layer_4/output_layer_norm/batchnorm/addAddV2?transformer/layer_4/output_layer_norm/moments/variance:output:0>transformer/layer_4/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_4/output_layer_norm/batchnorm/addļ 5transformer/layer_4/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_4/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_4/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_4/output_layer_norm/batchnorm/mulMul9transformer/layer_4/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_4/output_layer_norm/batchnorm/mul 5transformer/layer_4/output_layer_norm/batchnorm/mul_1Multransformer/layer_4/add_1:z:07transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_4/output_layer_norm/batchnorm/mul_1« 5transformer/layer_4/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_4/output_layer_norm/moments/mean:output:07transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_4/output_layer_norm/batchnorm/mul_2… >transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_4_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_4/output_layer_norm/batchnorm/subSubFtransformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_4/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_4/output_layer_norm/batchnorm/sub« 5transformer/layer_4/output_layer_norm/batchnorm/add_1AddV29transformer/layer_4/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_4/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_4/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_5/self_attention/query/einsum/EinsumEinsum9transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_5/self_attention/query/einsum/Einsum’ ;transformer/layer_5/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_5_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_5/self_attention/query/add/ReadVariableOp® ,transformer/layer_5/self_attention/query/addAddV2?transformer/layer_5/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_5/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_5/self_attention/query/addœ Ctransformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_5/self_attention/key/einsum/EinsumEinsum9transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_5/self_attention/key/einsum/Einsumł 9transformer/layer_5/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_5_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_5/self_attention/key/add/ReadVariableOp¦ *transformer/layer_5/self_attention/key/addAddV2=transformer/layer_5/self_attention/key/einsum/Einsum:output:0Atransformer/layer_5/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_5/self_attention/key/add¢ Etransformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_5/self_attention/value/einsum/EinsumEinsum9transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_5/self_attention/value/einsum/Einsum’ ;transformer/layer_5/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_5_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_5/self_attention/value/add/ReadVariableOp® ,transformer/layer_5/self_attention/value/addAddV2?transformer/layer_5/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_5/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_5/self_attention/value/addÄ 0transformer/layer_5/self_attention/einsum/EinsumEinsum.transformer/layer_5/self_attention/key/add:z:00transformer/layer_5/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_5/self_attention/einsum/Einsum™ (transformer/layer_5/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_5/self_attention/Mul/y‘ &transformer/layer_5/self_attention/MulMul9transformer/layer_5/self_attention/einsum/Einsum:output:01transformer/layer_5/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_5/self_attention/MulŅ Btransformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dimÄ >transformer/layer_5/self_attention/masked_softmax_5/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_5/self_attention/masked_softmax_5/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_5/self_attention/masked_softmax_5/ExpandDims» 9transformer/layer_5/self_attention/masked_softmax_5/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_5/self_attention/masked_softmax_5/sub/xŅ 7transformer/layer_5/self_attention/masked_softmax_5/subSubBtransformer/layer_5/self_attention/masked_softmax_5/sub/x:output:0Gtransformer/layer_5/self_attention/masked_softmax_5/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_5/self_attention/masked_softmax_5/sub» 9transformer/layer_5/self_attention/masked_softmax_5/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_5/self_attention/masked_softmax_5/mul/yĘ 7transformer/layer_5/self_attention/masked_softmax_5/mulMul;transformer/layer_5/self_attention/masked_softmax_5/sub:z:0Btransformer/layer_5/self_attention/masked_softmax_5/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_5/self_attention/masked_softmax_5/mul° 7transformer/layer_5/self_attention/masked_softmax_5/addAddV2*transformer/layer_5/self_attention/Mul:z:0;transformer/layer_5/self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_5/self_attention/masked_softmax_5/addŽ ;transformer/layer_5/self_attention/masked_softmax_5/SoftmaxSoftmax;transformer/layer_5/self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_5/self_attention/masked_softmax_5/Softmaxæ ;transformer/layer_5/self_attention/dropout_15/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2= ;transformer/layer_5/self_attention/dropout_15/dropout/ConstÖ 9transformer/layer_5/self_attention/dropout_15/dropout/MulMulEtransformer/layer_5/self_attention/masked_softmax_5/Softmax:softmax:0Dtransformer/layer_5/self_attention/dropout_15/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_5/self_attention/dropout_15/dropout/Mulļ ;transformer/layer_5/self_attention/dropout_15/dropout/ShapeShapeEtransformer/layer_5/self_attention/masked_softmax_5/Softmax:softmax:0* T0* _output_shapes :2= ;transformer/layer_5/self_attention/dropout_15/dropout/ShapeŲ Rtransformer/layer_5/self_attention/dropout_15/dropout/random_uniform/RandomUniform RandomUniformDtransformer/layer_5/self_attention/dropout_15/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02T Rtransformer/layer_5/self_attention/dropout_15/dropout/random_uniform/RandomUniformŃ Dtransformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2F Dtransformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual/y Btransformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual GreaterEqual[transformer/layer_5/self_attention/dropout_15/dropout/random_uniform/RandomUniform:output:0Mtransformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual£ :transformer/layer_5/self_attention/dropout_15/dropout/CastCastFtransformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_5/self_attention/dropout_15/dropout/CastĢ ;transformer/layer_5/self_attention/dropout_15/dropout/Mul_1Mul=transformer/layer_5/self_attention/dropout_15/dropout/Mul:z:0>transformer/layer_5/self_attention/dropout_15/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_5/self_attention/dropout_15/dropout/Mul_1Š 2transformer/layer_5/self_attention/einsum_1/EinsumEinsum?transformer/layer_5/self_attention/dropout_15/dropout/Mul_1:z:00transformer/layer_5/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_5/self_attention/einsum_1/EinsumĆ Ptransformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_5/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_5/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_5/self_attention/attention_output/einsum/Einsum Ftransformer/layer_5/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_5_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_5/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_5/self_attention/attention_output/addAddV2Jtransformer/layer_5/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_5/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_5/self_attention/attention_output/add” ,transformer/layer_5/dropout_16/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_5/dropout_16/dropout/Const“ *transformer/layer_5/dropout_16/dropout/MulMul;transformer/layer_5/self_attention/attention_output/add:z:05transformer/layer_5/dropout_16/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_5/dropout_16/dropout/MulĒ ,transformer/layer_5/dropout_16/dropout/ShapeShape;transformer/layer_5/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_5/dropout_16/dropout/ShapeŸ Ctransformer/layer_5/dropout_16/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_5/dropout_16/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_5/dropout_16/dropout/random_uniform/RandomUniform³ 5transformer/layer_5/dropout_16/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_5/dropout_16/dropout/GreaterEqual/yČ 3transformer/layer_5/dropout_16/dropout/GreaterEqual GreaterEqualLtransformer/layer_5/dropout_16/dropout/random_uniform/RandomUniform:output:0>transformer/layer_5/dropout_16/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_5/dropout_16/dropout/GreaterEqualź +transformer/layer_5/dropout_16/dropout/CastCast7transformer/layer_5/dropout_16/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_5/dropout_16/dropout/Cast„ ,transformer/layer_5/dropout_16/dropout/Mul_1Mul.transformer/layer_5/dropout_16/dropout/Mul:z:0/transformer/layer_5/dropout_16/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_5/dropout_16/dropout/Mul_1č transformer/layer_5/addAddV29transformer/layer_4/output_layer_norm/batchnorm/add_1:z:00transformer/layer_5/dropout_16/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_5/addę Ltransformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_5/self_attention_layer_norm/moments/meanMeantransformer/layer_5/add:z:0Utransformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_5/self_attention_layer_norm/moments/meanœ Btransformer/layer_5/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_5/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_5/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_5/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_5/add:z:0Ktransformer/layer_5/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_5/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_5/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_5/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_5/self_attention_layer_norm/moments/varianceĆ =transformer/layer_5/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_5/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_5/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_5/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_5/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_5/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_5/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_5/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_5/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_5/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_5/add:z:0?transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_5/self_attention_layer_norm/moments/mean:output:0?transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_5/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_5/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_5/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_5/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_5/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_5/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_5/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_5_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_5/intermediate/einsum/EinsumEinsumAtransformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_5/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_5/intermediate/einsum/Einsumä 3transformer/layer_5/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_5/output/einsum/EinsumŅ -transformer/layer_5/output/add/ReadVariableOpReadVariableOp6transformer_layer_5_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_5/output/add/ReadVariableOpó transformer/layer_5/output/addAddV21transformer/layer_5/output/einsum/Einsum:output:05transformer/layer_5/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_5/output/add” ,transformer/layer_5/dropout_17/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_5/dropout_17/dropout/Constś *transformer/layer_5/dropout_17/dropout/MulMul"transformer/layer_5/output/add:z:05transformer/layer_5/dropout_17/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_5/dropout_17/dropout/Mul® ,transformer/layer_5/dropout_17/dropout/ShapeShape"transformer/layer_5/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_5/dropout_17/dropout/ShapeŸ Ctransformer/layer_5/dropout_17/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_5/dropout_17/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_5/dropout_17/dropout/random_uniform/RandomUniform³ 5transformer/layer_5/dropout_17/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_5/dropout_17/dropout/GreaterEqual/yČ 3transformer/layer_5/dropout_17/dropout/GreaterEqual GreaterEqualLtransformer/layer_5/dropout_17/dropout/random_uniform/RandomUniform:output:0>transformer/layer_5/dropout_17/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_5/dropout_17/dropout/GreaterEqualź +transformer/layer_5/dropout_17/dropout/CastCast7transformer/layer_5/dropout_17/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_5/dropout_17/dropout/Cast„ ,transformer/layer_5/dropout_17/dropout/Mul_1Mul.transformer/layer_5/dropout_17/dropout/Mul:z:0/transformer/layer_5/dropout_17/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_5/dropout_17/dropout/Mul_1ō transformer/layer_5/add_1AddV20transformer/layer_5/dropout_17/dropout/Mul_1:z:0Atransformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_5/add_1Ö Dtransformer/layer_5/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_5/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_5/output_layer_norm/moments/meanMeantransformer/layer_5/add_1:z:0Mtransformer/layer_5/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_5/output_layer_norm/moments/mean„ :transformer/layer_5/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_5/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_5/output_layer_norm/moments/StopGradient» ?transformer/layer_5/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_5/add_1:z:0Ctransformer/layer_5/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_5/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_5/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_5/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_5/output_layer_norm/moments/varianceMeanCtransformer/layer_5/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_5/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_5/output_layer_norm/moments/variance³ 5transformer/layer_5/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_5/output_layer_norm/batchnorm/add/y³ 3transformer/layer_5/output_layer_norm/batchnorm/addAddV2?transformer/layer_5/output_layer_norm/moments/variance:output:0>transformer/layer_5/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_5/output_layer_norm/batchnorm/addļ 5transformer/layer_5/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_5/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_5/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_5/output_layer_norm/batchnorm/mulMul9transformer/layer_5/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_5/output_layer_norm/batchnorm/mul 5transformer/layer_5/output_layer_norm/batchnorm/mul_1Multransformer/layer_5/add_1:z:07transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_5/output_layer_norm/batchnorm/mul_1« 5transformer/layer_5/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_5/output_layer_norm/moments/mean:output:07transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_5/output_layer_norm/batchnorm/mul_2… >transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_5_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_5/output_layer_norm/batchnorm/subSubFtransformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_5/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_5/output_layer_norm/batchnorm/sub« 5transformer/layer_5/output_layer_norm/batchnorm/add_1AddV29transformer/layer_5/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_5/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_5/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_6/self_attention/query/einsum/EinsumEinsum9transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_6/self_attention/query/einsum/Einsum’ ;transformer/layer_6/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_6_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_6/self_attention/query/add/ReadVariableOp® ,transformer/layer_6/self_attention/query/addAddV2?transformer/layer_6/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_6/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_6/self_attention/query/addœ Ctransformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_6/self_attention/key/einsum/EinsumEinsum9transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_6/self_attention/key/einsum/Einsumł 9transformer/layer_6/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_6_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_6/self_attention/key/add/ReadVariableOp¦ *transformer/layer_6/self_attention/key/addAddV2=transformer/layer_6/self_attention/key/einsum/Einsum:output:0Atransformer/layer_6/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_6/self_attention/key/add¢ Etransformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_6/self_attention/value/einsum/EinsumEinsum9transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_6/self_attention/value/einsum/Einsum’ ;transformer/layer_6/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_6_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_6/self_attention/value/add/ReadVariableOp® ,transformer/layer_6/self_attention/value/addAddV2?transformer/layer_6/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_6/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_6/self_attention/value/addÄ 0transformer/layer_6/self_attention/einsum/EinsumEinsum.transformer/layer_6/self_attention/key/add:z:00transformer/layer_6/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_6/self_attention/einsum/Einsum™ (transformer/layer_6/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_6/self_attention/Mul/y‘ &transformer/layer_6/self_attention/MulMul9transformer/layer_6/self_attention/einsum/Einsum:output:01transformer/layer_6/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_6/self_attention/MulŅ Btransformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dimÄ >transformer/layer_6/self_attention/masked_softmax_6/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_6/self_attention/masked_softmax_6/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_6/self_attention/masked_softmax_6/ExpandDims» 9transformer/layer_6/self_attention/masked_softmax_6/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_6/self_attention/masked_softmax_6/sub/xŅ 7transformer/layer_6/self_attention/masked_softmax_6/subSubBtransformer/layer_6/self_attention/masked_softmax_6/sub/x:output:0Gtransformer/layer_6/self_attention/masked_softmax_6/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_6/self_attention/masked_softmax_6/sub» 9transformer/layer_6/self_attention/masked_softmax_6/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_6/self_attention/masked_softmax_6/mul/yĘ 7transformer/layer_6/self_attention/masked_softmax_6/mulMul;transformer/layer_6/self_attention/masked_softmax_6/sub:z:0Btransformer/layer_6/self_attention/masked_softmax_6/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_6/self_attention/masked_softmax_6/mul° 7transformer/layer_6/self_attention/masked_softmax_6/addAddV2*transformer/layer_6/self_attention/Mul:z:0;transformer/layer_6/self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_6/self_attention/masked_softmax_6/addŽ ;transformer/layer_6/self_attention/masked_softmax_6/SoftmaxSoftmax;transformer/layer_6/self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_6/self_attention/masked_softmax_6/Softmaxæ ;transformer/layer_6/self_attention/dropout_18/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2= ;transformer/layer_6/self_attention/dropout_18/dropout/ConstÖ 9transformer/layer_6/self_attention/dropout_18/dropout/MulMulEtransformer/layer_6/self_attention/masked_softmax_6/Softmax:softmax:0Dtransformer/layer_6/self_attention/dropout_18/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_6/self_attention/dropout_18/dropout/Mulļ ;transformer/layer_6/self_attention/dropout_18/dropout/ShapeShapeEtransformer/layer_6/self_attention/masked_softmax_6/Softmax:softmax:0* T0* _output_shapes :2= ;transformer/layer_6/self_attention/dropout_18/dropout/ShapeŲ Rtransformer/layer_6/self_attention/dropout_18/dropout/random_uniform/RandomUniform RandomUniformDtransformer/layer_6/self_attention/dropout_18/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02T Rtransformer/layer_6/self_attention/dropout_18/dropout/random_uniform/RandomUniformŃ Dtransformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2F Dtransformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual/y Btransformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual GreaterEqual[transformer/layer_6/self_attention/dropout_18/dropout/random_uniform/RandomUniform:output:0Mtransformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual£ :transformer/layer_6/self_attention/dropout_18/dropout/CastCastFtransformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_6/self_attention/dropout_18/dropout/CastĢ ;transformer/layer_6/self_attention/dropout_18/dropout/Mul_1Mul=transformer/layer_6/self_attention/dropout_18/dropout/Mul:z:0>transformer/layer_6/self_attention/dropout_18/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_6/self_attention/dropout_18/dropout/Mul_1Š 2transformer/layer_6/self_attention/einsum_1/EinsumEinsum?transformer/layer_6/self_attention/dropout_18/dropout/Mul_1:z:00transformer/layer_6/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_6/self_attention/einsum_1/EinsumĆ Ptransformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_6/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_6/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_6/self_attention/attention_output/einsum/Einsum Ftransformer/layer_6/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_6_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_6/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_6/self_attention/attention_output/addAddV2Jtransformer/layer_6/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_6/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_6/self_attention/attention_output/add” ,transformer/layer_6/dropout_19/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_6/dropout_19/dropout/Const“ *transformer/layer_6/dropout_19/dropout/MulMul;transformer/layer_6/self_attention/attention_output/add:z:05transformer/layer_6/dropout_19/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_6/dropout_19/dropout/MulĒ ,transformer/layer_6/dropout_19/dropout/ShapeShape;transformer/layer_6/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_6/dropout_19/dropout/ShapeŸ Ctransformer/layer_6/dropout_19/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_6/dropout_19/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_6/dropout_19/dropout/random_uniform/RandomUniform³ 5transformer/layer_6/dropout_19/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_6/dropout_19/dropout/GreaterEqual/yČ 3transformer/layer_6/dropout_19/dropout/GreaterEqual GreaterEqualLtransformer/layer_6/dropout_19/dropout/random_uniform/RandomUniform:output:0>transformer/layer_6/dropout_19/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_6/dropout_19/dropout/GreaterEqualź +transformer/layer_6/dropout_19/dropout/CastCast7transformer/layer_6/dropout_19/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_6/dropout_19/dropout/Cast„ ,transformer/layer_6/dropout_19/dropout/Mul_1Mul.transformer/layer_6/dropout_19/dropout/Mul:z:0/transformer/layer_6/dropout_19/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_6/dropout_19/dropout/Mul_1č transformer/layer_6/addAddV29transformer/layer_5/output_layer_norm/batchnorm/add_1:z:00transformer/layer_6/dropout_19/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_6/addę Ltransformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_6/self_attention_layer_norm/moments/meanMeantransformer/layer_6/add:z:0Utransformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_6/self_attention_layer_norm/moments/meanœ Btransformer/layer_6/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_6/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_6/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_6/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_6/add:z:0Ktransformer/layer_6/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_6/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_6/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_6/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_6/self_attention_layer_norm/moments/varianceĆ =transformer/layer_6/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_6/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_6/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_6/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_6/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_6/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_6/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_6/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_6/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_6/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_6/add:z:0?transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_6/self_attention_layer_norm/moments/mean:output:0?transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_6/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_6/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_6/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_6/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_6/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_6/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_6/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_6_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_6/intermediate/einsum/EinsumEinsumAtransformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_6/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_6/intermediate/einsum/Einsumä 3transformer/layer_6/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_6/output/einsum/EinsumŅ -transformer/layer_6/output/add/ReadVariableOpReadVariableOp6transformer_layer_6_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_6/output/add/ReadVariableOpó transformer/layer_6/output/addAddV21transformer/layer_6/output/einsum/Einsum:output:05transformer/layer_6/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_6/output/add” ,transformer/layer_6/dropout_20/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_6/dropout_20/dropout/Constś *transformer/layer_6/dropout_20/dropout/MulMul"transformer/layer_6/output/add:z:05transformer/layer_6/dropout_20/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_6/dropout_20/dropout/Mul® ,transformer/layer_6/dropout_20/dropout/ShapeShape"transformer/layer_6/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_6/dropout_20/dropout/ShapeŸ Ctransformer/layer_6/dropout_20/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_6/dropout_20/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_6/dropout_20/dropout/random_uniform/RandomUniform³ 5transformer/layer_6/dropout_20/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_6/dropout_20/dropout/GreaterEqual/yČ 3transformer/layer_6/dropout_20/dropout/GreaterEqual GreaterEqualLtransformer/layer_6/dropout_20/dropout/random_uniform/RandomUniform:output:0>transformer/layer_6/dropout_20/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_6/dropout_20/dropout/GreaterEqualź +transformer/layer_6/dropout_20/dropout/CastCast7transformer/layer_6/dropout_20/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_6/dropout_20/dropout/Cast„ ,transformer/layer_6/dropout_20/dropout/Mul_1Mul.transformer/layer_6/dropout_20/dropout/Mul:z:0/transformer/layer_6/dropout_20/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_6/dropout_20/dropout/Mul_1ō transformer/layer_6/add_1AddV20transformer/layer_6/dropout_20/dropout/Mul_1:z:0Atransformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_6/add_1Ö Dtransformer/layer_6/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_6/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_6/output_layer_norm/moments/meanMeantransformer/layer_6/add_1:z:0Mtransformer/layer_6/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_6/output_layer_norm/moments/mean„ :transformer/layer_6/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_6/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_6/output_layer_norm/moments/StopGradient» ?transformer/layer_6/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_6/add_1:z:0Ctransformer/layer_6/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_6/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_6/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_6/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_6/output_layer_norm/moments/varianceMeanCtransformer/layer_6/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_6/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_6/output_layer_norm/moments/variance³ 5transformer/layer_6/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_6/output_layer_norm/batchnorm/add/y³ 3transformer/layer_6/output_layer_norm/batchnorm/addAddV2?transformer/layer_6/output_layer_norm/moments/variance:output:0>transformer/layer_6/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_6/output_layer_norm/batchnorm/addļ 5transformer/layer_6/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_6/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_6/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_6/output_layer_norm/batchnorm/mulMul9transformer/layer_6/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_6/output_layer_norm/batchnorm/mul 5transformer/layer_6/output_layer_norm/batchnorm/mul_1Multransformer/layer_6/add_1:z:07transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_6/output_layer_norm/batchnorm/mul_1« 5transformer/layer_6/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_6/output_layer_norm/moments/mean:output:07transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_6/output_layer_norm/batchnorm/mul_2… >transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_6_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_6/output_layer_norm/batchnorm/subSubFtransformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_6/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_6/output_layer_norm/batchnorm/sub« 5transformer/layer_6/output_layer_norm/batchnorm/add_1AddV29transformer/layer_6/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_6/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_6/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_7/self_attention/query/einsum/EinsumEinsum9transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_7/self_attention/query/einsum/Einsum’ ;transformer/layer_7/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_7_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_7/self_attention/query/add/ReadVariableOp® ,transformer/layer_7/self_attention/query/addAddV2?transformer/layer_7/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_7/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_7/self_attention/query/addœ Ctransformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_7/self_attention/key/einsum/EinsumEinsum9transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_7/self_attention/key/einsum/Einsumł 9transformer/layer_7/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_7_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_7/self_attention/key/add/ReadVariableOp¦ *transformer/layer_7/self_attention/key/addAddV2=transformer/layer_7/self_attention/key/einsum/Einsum:output:0Atransformer/layer_7/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_7/self_attention/key/add¢ Etransformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_7/self_attention/value/einsum/EinsumEinsum9transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_7/self_attention/value/einsum/Einsum’ ;transformer/layer_7/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_7_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_7/self_attention/value/add/ReadVariableOp® ,transformer/layer_7/self_attention/value/addAddV2?transformer/layer_7/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_7/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_7/self_attention/value/addÄ 0transformer/layer_7/self_attention/einsum/EinsumEinsum.transformer/layer_7/self_attention/key/add:z:00transformer/layer_7/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_7/self_attention/einsum/Einsum™ (transformer/layer_7/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_7/self_attention/Mul/y‘ &transformer/layer_7/self_attention/MulMul9transformer/layer_7/self_attention/einsum/Einsum:output:01transformer/layer_7/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_7/self_attention/MulŅ Btransformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dimÄ >transformer/layer_7/self_attention/masked_softmax_7/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_7/self_attention/masked_softmax_7/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_7/self_attention/masked_softmax_7/ExpandDims» 9transformer/layer_7/self_attention/masked_softmax_7/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_7/self_attention/masked_softmax_7/sub/xŅ 7transformer/layer_7/self_attention/masked_softmax_7/subSubBtransformer/layer_7/self_attention/masked_softmax_7/sub/x:output:0Gtransformer/layer_7/self_attention/masked_softmax_7/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_7/self_attention/masked_softmax_7/sub» 9transformer/layer_7/self_attention/masked_softmax_7/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_7/self_attention/masked_softmax_7/mul/yĘ 7transformer/layer_7/self_attention/masked_softmax_7/mulMul;transformer/layer_7/self_attention/masked_softmax_7/sub:z:0Btransformer/layer_7/self_attention/masked_softmax_7/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_7/self_attention/masked_softmax_7/mul° 7transformer/layer_7/self_attention/masked_softmax_7/addAddV2*transformer/layer_7/self_attention/Mul:z:0;transformer/layer_7/self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_7/self_attention/masked_softmax_7/addŽ ;transformer/layer_7/self_attention/masked_softmax_7/SoftmaxSoftmax;transformer/layer_7/self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_7/self_attention/masked_softmax_7/Softmaxæ ;transformer/layer_7/self_attention/dropout_21/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2= ;transformer/layer_7/self_attention/dropout_21/dropout/ConstÖ 9transformer/layer_7/self_attention/dropout_21/dropout/MulMulEtransformer/layer_7/self_attention/masked_softmax_7/Softmax:softmax:0Dtransformer/layer_7/self_attention/dropout_21/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_7/self_attention/dropout_21/dropout/Mulļ ;transformer/layer_7/self_attention/dropout_21/dropout/ShapeShapeEtransformer/layer_7/self_attention/masked_softmax_7/Softmax:softmax:0* T0* _output_shapes :2= ;transformer/layer_7/self_attention/dropout_21/dropout/ShapeŲ Rtransformer/layer_7/self_attention/dropout_21/dropout/random_uniform/RandomUniform RandomUniformDtransformer/layer_7/self_attention/dropout_21/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02T Rtransformer/layer_7/self_attention/dropout_21/dropout/random_uniform/RandomUniformŃ Dtransformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2F Dtransformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual/y Btransformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual GreaterEqual[transformer/layer_7/self_attention/dropout_21/dropout/random_uniform/RandomUniform:output:0Mtransformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual£ :transformer/layer_7/self_attention/dropout_21/dropout/CastCastFtransformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_7/self_attention/dropout_21/dropout/CastĢ ;transformer/layer_7/self_attention/dropout_21/dropout/Mul_1Mul=transformer/layer_7/self_attention/dropout_21/dropout/Mul:z:0>transformer/layer_7/self_attention/dropout_21/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_7/self_attention/dropout_21/dropout/Mul_1Š 2transformer/layer_7/self_attention/einsum_1/EinsumEinsum?transformer/layer_7/self_attention/dropout_21/dropout/Mul_1:z:00transformer/layer_7/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_7/self_attention/einsum_1/EinsumĆ Ptransformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_7/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_7/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_7/self_attention/attention_output/einsum/Einsum Ftransformer/layer_7/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_7_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_7/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_7/self_attention/attention_output/addAddV2Jtransformer/layer_7/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_7/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_7/self_attention/attention_output/add” ,transformer/layer_7/dropout_22/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_7/dropout_22/dropout/Const“ *transformer/layer_7/dropout_22/dropout/MulMul;transformer/layer_7/self_attention/attention_output/add:z:05transformer/layer_7/dropout_22/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_7/dropout_22/dropout/MulĒ ,transformer/layer_7/dropout_22/dropout/ShapeShape;transformer/layer_7/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_7/dropout_22/dropout/ShapeŸ Ctransformer/layer_7/dropout_22/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_7/dropout_22/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_7/dropout_22/dropout/random_uniform/RandomUniform³ 5transformer/layer_7/dropout_22/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_7/dropout_22/dropout/GreaterEqual/yČ 3transformer/layer_7/dropout_22/dropout/GreaterEqual GreaterEqualLtransformer/layer_7/dropout_22/dropout/random_uniform/RandomUniform:output:0>transformer/layer_7/dropout_22/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_7/dropout_22/dropout/GreaterEqualź +transformer/layer_7/dropout_22/dropout/CastCast7transformer/layer_7/dropout_22/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_7/dropout_22/dropout/Cast„ ,transformer/layer_7/dropout_22/dropout/Mul_1Mul.transformer/layer_7/dropout_22/dropout/Mul:z:0/transformer/layer_7/dropout_22/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_7/dropout_22/dropout/Mul_1č transformer/layer_7/addAddV29transformer/layer_6/output_layer_norm/batchnorm/add_1:z:00transformer/layer_7/dropout_22/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_7/addę Ltransformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_7/self_attention_layer_norm/moments/meanMeantransformer/layer_7/add:z:0Utransformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_7/self_attention_layer_norm/moments/meanœ Btransformer/layer_7/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_7/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_7/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_7/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_7/add:z:0Ktransformer/layer_7/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_7/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_7/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_7/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_7/self_attention_layer_norm/moments/varianceĆ =transformer/layer_7/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_7/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_7/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_7/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_7/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_7/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_7/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_7/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_7/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_7/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_7/add:z:0?transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_7/self_attention_layer_norm/moments/mean:output:0?transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_7/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_7/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_7/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_7/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_7/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_7/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_7/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_7_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_7/intermediate/einsum/EinsumEinsumAtransformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_7/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_7/intermediate/einsum/Einsumä 3transformer/layer_7/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_7/output/einsum/EinsumŅ -transformer/layer_7/output/add/ReadVariableOpReadVariableOp6transformer_layer_7_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_7/output/add/ReadVariableOpó transformer/layer_7/output/addAddV21transformer/layer_7/output/einsum/Einsum:output:05transformer/layer_7/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_7/output/add” ,transformer/layer_7/dropout_23/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_7/dropout_23/dropout/Constś *transformer/layer_7/dropout_23/dropout/MulMul"transformer/layer_7/output/add:z:05transformer/layer_7/dropout_23/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_7/dropout_23/dropout/Mul® ,transformer/layer_7/dropout_23/dropout/ShapeShape"transformer/layer_7/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_7/dropout_23/dropout/ShapeŸ Ctransformer/layer_7/dropout_23/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_7/dropout_23/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_7/dropout_23/dropout/random_uniform/RandomUniform³ 5transformer/layer_7/dropout_23/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_7/dropout_23/dropout/GreaterEqual/yČ 3transformer/layer_7/dropout_23/dropout/GreaterEqual GreaterEqualLtransformer/layer_7/dropout_23/dropout/random_uniform/RandomUniform:output:0>transformer/layer_7/dropout_23/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_7/dropout_23/dropout/GreaterEqualź +transformer/layer_7/dropout_23/dropout/CastCast7transformer/layer_7/dropout_23/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_7/dropout_23/dropout/Cast„ ,transformer/layer_7/dropout_23/dropout/Mul_1Mul.transformer/layer_7/dropout_23/dropout/Mul:z:0/transformer/layer_7/dropout_23/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_7/dropout_23/dropout/Mul_1ō transformer/layer_7/add_1AddV20transformer/layer_7/dropout_23/dropout/Mul_1:z:0Atransformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_7/add_1Ö Dtransformer/layer_7/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_7/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_7/output_layer_norm/moments/meanMeantransformer/layer_7/add_1:z:0Mtransformer/layer_7/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_7/output_layer_norm/moments/mean„ :transformer/layer_7/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_7/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_7/output_layer_norm/moments/StopGradient» ?transformer/layer_7/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_7/add_1:z:0Ctransformer/layer_7/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_7/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_7/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_7/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_7/output_layer_norm/moments/varianceMeanCtransformer/layer_7/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_7/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_7/output_layer_norm/moments/variance³ 5transformer/layer_7/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_7/output_layer_norm/batchnorm/add/y³ 3transformer/layer_7/output_layer_norm/batchnorm/addAddV2?transformer/layer_7/output_layer_norm/moments/variance:output:0>transformer/layer_7/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_7/output_layer_norm/batchnorm/addļ 5transformer/layer_7/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_7/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_7/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_7/output_layer_norm/batchnorm/mulMul9transformer/layer_7/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_7/output_layer_norm/batchnorm/mul 5transformer/layer_7/output_layer_norm/batchnorm/mul_1Multransformer/layer_7/add_1:z:07transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_7/output_layer_norm/batchnorm/mul_1« 5transformer/layer_7/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_7/output_layer_norm/moments/mean:output:07transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_7/output_layer_norm/batchnorm/mul_2… >transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_7_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_7/output_layer_norm/batchnorm/subSubFtransformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_7/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_7/output_layer_norm/batchnorm/sub« 5transformer/layer_7/output_layer_norm/batchnorm/add_1AddV29transformer/layer_7/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_7/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_7/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_8/self_attention/query/einsum/EinsumEinsum9transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_8/self_attention/query/einsum/Einsum’ ;transformer/layer_8/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_8_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_8/self_attention/query/add/ReadVariableOp® ,transformer/layer_8/self_attention/query/addAddV2?transformer/layer_8/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_8/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_8/self_attention/query/addœ Ctransformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_8/self_attention/key/einsum/EinsumEinsum9transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_8/self_attention/key/einsum/Einsumł 9transformer/layer_8/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_8_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_8/self_attention/key/add/ReadVariableOp¦ *transformer/layer_8/self_attention/key/addAddV2=transformer/layer_8/self_attention/key/einsum/Einsum:output:0Atransformer/layer_8/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_8/self_attention/key/add¢ Etransformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_8/self_attention/value/einsum/EinsumEinsum9transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_8/self_attention/value/einsum/Einsum’ ;transformer/layer_8/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_8_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_8/self_attention/value/add/ReadVariableOp® ,transformer/layer_8/self_attention/value/addAddV2?transformer/layer_8/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_8/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_8/self_attention/value/addÄ 0transformer/layer_8/self_attention/einsum/EinsumEinsum.transformer/layer_8/self_attention/key/add:z:00transformer/layer_8/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_8/self_attention/einsum/Einsum™ (transformer/layer_8/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_8/self_attention/Mul/y‘ &transformer/layer_8/self_attention/MulMul9transformer/layer_8/self_attention/einsum/Einsum:output:01transformer/layer_8/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_8/self_attention/MulŅ Btransformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dimÄ >transformer/layer_8/self_attention/masked_softmax_8/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_8/self_attention/masked_softmax_8/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_8/self_attention/masked_softmax_8/ExpandDims» 9transformer/layer_8/self_attention/masked_softmax_8/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_8/self_attention/masked_softmax_8/sub/xŅ 7transformer/layer_8/self_attention/masked_softmax_8/subSubBtransformer/layer_8/self_attention/masked_softmax_8/sub/x:output:0Gtransformer/layer_8/self_attention/masked_softmax_8/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_8/self_attention/masked_softmax_8/sub» 9transformer/layer_8/self_attention/masked_softmax_8/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_8/self_attention/masked_softmax_8/mul/yĘ 7transformer/layer_8/self_attention/masked_softmax_8/mulMul;transformer/layer_8/self_attention/masked_softmax_8/sub:z:0Btransformer/layer_8/self_attention/masked_softmax_8/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_8/self_attention/masked_softmax_8/mul° 7transformer/layer_8/self_attention/masked_softmax_8/addAddV2*transformer/layer_8/self_attention/Mul:z:0;transformer/layer_8/self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_8/self_attention/masked_softmax_8/addŽ ;transformer/layer_8/self_attention/masked_softmax_8/SoftmaxSoftmax;transformer/layer_8/self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_8/self_attention/masked_softmax_8/Softmaxæ ;transformer/layer_8/self_attention/dropout_24/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2= ;transformer/layer_8/self_attention/dropout_24/dropout/ConstÖ 9transformer/layer_8/self_attention/dropout_24/dropout/MulMulEtransformer/layer_8/self_attention/masked_softmax_8/Softmax:softmax:0Dtransformer/layer_8/self_attention/dropout_24/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_8/self_attention/dropout_24/dropout/Mulļ ;transformer/layer_8/self_attention/dropout_24/dropout/ShapeShapeEtransformer/layer_8/self_attention/masked_softmax_8/Softmax:softmax:0* T0* _output_shapes :2= ;transformer/layer_8/self_attention/dropout_24/dropout/ShapeŲ Rtransformer/layer_8/self_attention/dropout_24/dropout/random_uniform/RandomUniform RandomUniformDtransformer/layer_8/self_attention/dropout_24/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02T Rtransformer/layer_8/self_attention/dropout_24/dropout/random_uniform/RandomUniformŃ Dtransformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2F Dtransformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual/y Btransformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual GreaterEqual[transformer/layer_8/self_attention/dropout_24/dropout/random_uniform/RandomUniform:output:0Mtransformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual£ :transformer/layer_8/self_attention/dropout_24/dropout/CastCastFtransformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_8/self_attention/dropout_24/dropout/CastĢ ;transformer/layer_8/self_attention/dropout_24/dropout/Mul_1Mul=transformer/layer_8/self_attention/dropout_24/dropout/Mul:z:0>transformer/layer_8/self_attention/dropout_24/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_8/self_attention/dropout_24/dropout/Mul_1Š 2transformer/layer_8/self_attention/einsum_1/EinsumEinsum?transformer/layer_8/self_attention/dropout_24/dropout/Mul_1:z:00transformer/layer_8/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_8/self_attention/einsum_1/EinsumĆ Ptransformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_8/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_8/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_8/self_attention/attention_output/einsum/Einsum Ftransformer/layer_8/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_8_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_8/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_8/self_attention/attention_output/addAddV2Jtransformer/layer_8/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_8/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_8/self_attention/attention_output/add” ,transformer/layer_8/dropout_25/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_8/dropout_25/dropout/Const“ *transformer/layer_8/dropout_25/dropout/MulMul;transformer/layer_8/self_attention/attention_output/add:z:05transformer/layer_8/dropout_25/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_8/dropout_25/dropout/MulĒ ,transformer/layer_8/dropout_25/dropout/ShapeShape;transformer/layer_8/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_8/dropout_25/dropout/ShapeŸ Ctransformer/layer_8/dropout_25/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_8/dropout_25/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_8/dropout_25/dropout/random_uniform/RandomUniform³ 5transformer/layer_8/dropout_25/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_8/dropout_25/dropout/GreaterEqual/yČ 3transformer/layer_8/dropout_25/dropout/GreaterEqual GreaterEqualLtransformer/layer_8/dropout_25/dropout/random_uniform/RandomUniform:output:0>transformer/layer_8/dropout_25/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_8/dropout_25/dropout/GreaterEqualź +transformer/layer_8/dropout_25/dropout/CastCast7transformer/layer_8/dropout_25/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_8/dropout_25/dropout/Cast„ ,transformer/layer_8/dropout_25/dropout/Mul_1Mul.transformer/layer_8/dropout_25/dropout/Mul:z:0/transformer/layer_8/dropout_25/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_8/dropout_25/dropout/Mul_1č transformer/layer_8/addAddV29transformer/layer_7/output_layer_norm/batchnorm/add_1:z:00transformer/layer_8/dropout_25/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_8/addę Ltransformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_8/self_attention_layer_norm/moments/meanMeantransformer/layer_8/add:z:0Utransformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_8/self_attention_layer_norm/moments/meanœ Btransformer/layer_8/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_8/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_8/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_8/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_8/add:z:0Ktransformer/layer_8/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_8/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_8/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_8/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_8/self_attention_layer_norm/moments/varianceĆ =transformer/layer_8/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_8/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_8/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_8/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_8/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_8/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_8/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_8/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_8/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_8/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_8/add:z:0?transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_8/self_attention_layer_norm/moments/mean:output:0?transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_8/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_8/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_8/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_8/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_8/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_8/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_8/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_8_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_8/intermediate/einsum/EinsumEinsumAtransformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_8/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_8/intermediate/einsum/Einsumä 3transformer/layer_8/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_8/output/einsum/EinsumŅ -transformer/layer_8/output/add/ReadVariableOpReadVariableOp6transformer_layer_8_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_8/output/add/ReadVariableOpó transformer/layer_8/output/addAddV21transformer/layer_8/output/einsum/Einsum:output:05transformer/layer_8/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_8/output/add” ,transformer/layer_8/dropout_26/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_8/dropout_26/dropout/Constś *transformer/layer_8/dropout_26/dropout/MulMul"transformer/layer_8/output/add:z:05transformer/layer_8/dropout_26/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_8/dropout_26/dropout/Mul® ,transformer/layer_8/dropout_26/dropout/ShapeShape"transformer/layer_8/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_8/dropout_26/dropout/ShapeŸ Ctransformer/layer_8/dropout_26/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_8/dropout_26/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_8/dropout_26/dropout/random_uniform/RandomUniform³ 5transformer/layer_8/dropout_26/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_8/dropout_26/dropout/GreaterEqual/yČ 3transformer/layer_8/dropout_26/dropout/GreaterEqual GreaterEqualLtransformer/layer_8/dropout_26/dropout/random_uniform/RandomUniform:output:0>transformer/layer_8/dropout_26/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_8/dropout_26/dropout/GreaterEqualź +transformer/layer_8/dropout_26/dropout/CastCast7transformer/layer_8/dropout_26/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_8/dropout_26/dropout/Cast„ ,transformer/layer_8/dropout_26/dropout/Mul_1Mul.transformer/layer_8/dropout_26/dropout/Mul:z:0/transformer/layer_8/dropout_26/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_8/dropout_26/dropout/Mul_1ō transformer/layer_8/add_1AddV20transformer/layer_8/dropout_26/dropout/Mul_1:z:0Atransformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_8/add_1Ö Dtransformer/layer_8/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_8/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_8/output_layer_norm/moments/meanMeantransformer/layer_8/add_1:z:0Mtransformer/layer_8/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_8/output_layer_norm/moments/mean„ :transformer/layer_8/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_8/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_8/output_layer_norm/moments/StopGradient» ?transformer/layer_8/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_8/add_1:z:0Ctransformer/layer_8/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_8/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_8/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_8/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_8/output_layer_norm/moments/varianceMeanCtransformer/layer_8/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_8/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_8/output_layer_norm/moments/variance³ 5transformer/layer_8/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_8/output_layer_norm/batchnorm/add/y³ 3transformer/layer_8/output_layer_norm/batchnorm/addAddV2?transformer/layer_8/output_layer_norm/moments/variance:output:0>transformer/layer_8/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_8/output_layer_norm/batchnorm/addļ 5transformer/layer_8/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_8/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_8/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_8/output_layer_norm/batchnorm/mulMul9transformer/layer_8/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_8/output_layer_norm/batchnorm/mul 5transformer/layer_8/output_layer_norm/batchnorm/mul_1Multransformer/layer_8/add_1:z:07transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_8/output_layer_norm/batchnorm/mul_1« 5transformer/layer_8/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_8/output_layer_norm/moments/mean:output:07transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_8/output_layer_norm/batchnorm/mul_2… >transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_8_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_8/output_layer_norm/batchnorm/subSubFtransformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_8/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_8/output_layer_norm/batchnorm/sub« 5transformer/layer_8/output_layer_norm/batchnorm/add_1AddV29transformer/layer_8/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_8/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_8/output_layer_norm/batchnorm/add_1¢ Etransformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpķ 6transformer/layer_9/self_attention/query/einsum/EinsumEinsum9transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_9/self_attention/query/einsum/Einsum’ ;transformer/layer_9/self_attention/query/add/ReadVariableOpReadVariableOpDtransformer_layer_9_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_9/self_attention/query/add/ReadVariableOp® ,transformer/layer_9/self_attention/query/addAddV2?transformer/layer_9/self_attention/query/einsum/Einsum:output:0Ctransformer/layer_9/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_9/self_attention/query/addœ Ctransformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpLtransformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02E Ctransformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpē 4transformer/layer_9/self_attention/key/einsum/EinsumEinsum9transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Ktransformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde26 4transformer/layer_9/self_attention/key/einsum/Einsumł 9transformer/layer_9/self_attention/key/add/ReadVariableOpReadVariableOpBtransformer_layer_9_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02; 9transformer/layer_9/self_attention/key/add/ReadVariableOp¦ *transformer/layer_9/self_attention/key/addAddV2=transformer/layer_9/self_attention/key/einsum/Einsum:output:0Atransformer/layer_9/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2, *transformer/layer_9/self_attention/key/add¢ Etransformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpNtransformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02G Etransformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpķ 6transformer/layer_9/self_attention/value/einsum/EinsumEinsum9transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Mtransformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde28 6transformer/layer_9/self_attention/value/einsum/Einsum’ ;transformer/layer_9/self_attention/value/add/ReadVariableOpReadVariableOpDtransformer_layer_9_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02= ;transformer/layer_9/self_attention/value/add/ReadVariableOp® ,transformer/layer_9/self_attention/value/addAddV2?transformer/layer_9/self_attention/value/einsum/Einsum:output:0Ctransformer/layer_9/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2. ,transformer/layer_9/self_attention/value/addÄ 0transformer/layer_9/self_attention/einsum/EinsumEinsum.transformer/layer_9/self_attention/key/add:z:00transformer/layer_9/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe22 0transformer/layer_9/self_attention/einsum/Einsum™ (transformer/layer_9/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2* (transformer/layer_9/self_attention/Mul/y‘ &transformer/layer_9/self_attention/MulMul9transformer/layer_9/self_attention/einsum/Einsum:output:01transformer/layer_9/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &transformer/layer_9/self_attention/MulŅ Btransformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2D Btransformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dimÄ >transformer/layer_9/self_attention/masked_softmax_9/ExpandDims ExpandDimsself_attention_mask/mul:z:0Ktransformer/layer_9/self_attention/masked_softmax_9/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_9/self_attention/masked_softmax_9/ExpandDims» 9transformer/layer_9/self_attention/masked_softmax_9/sub/xConst* _output_shapes :* dtype0* value B *€?2; 9transformer/layer_9/self_attention/masked_softmax_9/sub/xŅ 7transformer/layer_9/self_attention/masked_softmax_9/subSubBtransformer/layer_9/self_attention/masked_softmax_9/sub/x:output:0Gtransformer/layer_9/self_attention/masked_softmax_9/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_9/self_attention/masked_softmax_9/sub» 9transformer/layer_9/self_attention/masked_softmax_9/mul/yConst* _output_shapes :* dtype0* value B *@Ę2; 9transformer/layer_9/self_attention/masked_softmax_9/mul/yĘ 7transformer/layer_9/self_attention/masked_softmax_9/mulMul;transformer/layer_9/self_attention/masked_softmax_9/sub:z:0Btransformer/layer_9/self_attention/masked_softmax_9/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_9/self_attention/masked_softmax_9/mul° 7transformer/layer_9/self_attention/masked_softmax_9/addAddV2*transformer/layer_9/self_attention/Mul:z:0;transformer/layer_9/self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’29 7transformer/layer_9/self_attention/masked_softmax_9/addŽ ;transformer/layer_9/self_attention/masked_softmax_9/SoftmaxSoftmax;transformer/layer_9/self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_9/self_attention/masked_softmax_9/Softmaxæ ;transformer/layer_9/self_attention/dropout_27/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2= ;transformer/layer_9/self_attention/dropout_27/dropout/ConstÖ 9transformer/layer_9/self_attention/dropout_27/dropout/MulMulEtransformer/layer_9/self_attention/masked_softmax_9/Softmax:softmax:0Dtransformer/layer_9/self_attention/dropout_27/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_9/self_attention/dropout_27/dropout/Mulļ ;transformer/layer_9/self_attention/dropout_27/dropout/ShapeShapeEtransformer/layer_9/self_attention/masked_softmax_9/Softmax:softmax:0* T0* _output_shapes :2= ;transformer/layer_9/self_attention/dropout_27/dropout/ShapeŲ Rtransformer/layer_9/self_attention/dropout_27/dropout/random_uniform/RandomUniform RandomUniformDtransformer/layer_9/self_attention/dropout_27/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02T Rtransformer/layer_9/self_attention/dropout_27/dropout/random_uniform/RandomUniformŃ Dtransformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2F Dtransformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual/y Btransformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual GreaterEqual[transformer/layer_9/self_attention/dropout_27/dropout/random_uniform/RandomUniform:output:0Mtransformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual£ :transformer/layer_9/self_attention/dropout_27/dropout/CastCastFtransformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_9/self_attention/dropout_27/dropout/CastĢ ;transformer/layer_9/self_attention/dropout_27/dropout/Mul_1Mul=transformer/layer_9/self_attention/dropout_27/dropout/Mul:z:0>transformer/layer_9/self_attention/dropout_27/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_9/self_attention/dropout_27/dropout/Mul_1Š 2transformer/layer_9/self_attention/einsum_1/EinsumEinsum?transformer/layer_9/self_attention/dropout_27/dropout/Mul_1:z:00transformer/layer_9/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd24 2transformer/layer_9/self_attention/einsum_1/EinsumĆ Ptransformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpYtransformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02R Ptransformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp Atransformer/layer_9/self_attention/attention_output/einsum/EinsumEinsum;transformer/layer_9/self_attention/einsum_1/Einsum:output:0Xtransformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2C Atransformer/layer_9/self_attention/attention_output/einsum/Einsum Ftransformer/layer_9/self_attention/attention_output/add/ReadVariableOpReadVariableOpOtransformer_layer_9_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_9/self_attention/attention_output/add/ReadVariableOp× 7transformer/layer_9/self_attention/attention_output/addAddV2Jtransformer/layer_9/self_attention/attention_output/einsum/Einsum:output:0Ntransformer/layer_9/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer/layer_9/self_attention/attention_output/add” ,transformer/layer_9/dropout_28/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_9/dropout_28/dropout/Const“ *transformer/layer_9/dropout_28/dropout/MulMul;transformer/layer_9/self_attention/attention_output/add:z:05transformer/layer_9/dropout_28/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_9/dropout_28/dropout/MulĒ ,transformer/layer_9/dropout_28/dropout/ShapeShape;transformer/layer_9/self_attention/attention_output/add:z:0* T0* _output_shapes :2. ,transformer/layer_9/dropout_28/dropout/ShapeŸ Ctransformer/layer_9/dropout_28/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_9/dropout_28/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_9/dropout_28/dropout/random_uniform/RandomUniform³ 5transformer/layer_9/dropout_28/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_9/dropout_28/dropout/GreaterEqual/yČ 3transformer/layer_9/dropout_28/dropout/GreaterEqual GreaterEqualLtransformer/layer_9/dropout_28/dropout/random_uniform/RandomUniform:output:0>transformer/layer_9/dropout_28/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_9/dropout_28/dropout/GreaterEqualź +transformer/layer_9/dropout_28/dropout/CastCast7transformer/layer_9/dropout_28/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_9/dropout_28/dropout/Cast„ ,transformer/layer_9/dropout_28/dropout/Mul_1Mul.transformer/layer_9/dropout_28/dropout/Mul:z:0/transformer/layer_9/dropout_28/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_9/dropout_28/dropout/Mul_1č transformer/layer_9/addAddV29transformer/layer_8/output_layer_norm/batchnorm/add_1:z:00transformer/layer_9/dropout_28/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_9/addę Ltransformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesÄ :transformer/layer_9/self_attention_layer_norm/moments/meanMeantransformer/layer_9/add:z:0Utransformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer/layer_9/self_attention_layer_norm/moments/meanœ Btransformer/layer_9/self_attention_layer_norm/moments/StopGradient StopGradientCtransformer/layer_9/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2D Btransformer/layer_9/self_attention_layer_norm/moments/StopGradientŃ Gtransformer/layer_9/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_9/add:z:0Ktransformer/layer_9/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer/layer_9/self_attention_layer_norm/moments/SquaredDifferenceī Ptransformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2R Ptransformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indices€ >transformer/layer_9/self_attention_layer_norm/moments/varianceMeanKtransformer/layer_9/self_attention_layer_norm/moments/SquaredDifference:z:0Ytransformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2@ >transformer/layer_9/self_attention_layer_norm/moments/varianceĆ =transformer/layer_9/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2? =transformer/layer_9/self_attention_layer_norm/batchnorm/add/yÓ ;transformer/layer_9/self_attention_layer_norm/batchnorm/addAddV2Gtransformer/layer_9/self_attention_layer_norm/moments/variance:output:0Ftransformer/layer_9/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2= ;transformer/layer_9/self_attention_layer_norm/batchnorm/add‡ =transformer/layer_9/self_attention_layer_norm/batchnorm/RsqrtRsqrt?transformer/layer_9/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrt© Jtransformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpStransformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02L Jtransformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpŲ ;transformer/layer_9/self_attention_layer_norm/batchnorm/mulMulAtransformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrt:y:0Rtransformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_9/self_attention_layer_norm/batchnorm/mul£ =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_9/add:z:0?transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1Ė =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2MulCtransformer/layer_9/self_attention_layer_norm/moments/mean:output:0?transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2 Ftransformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpOtransformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOpŌ ;transformer/layer_9/self_attention_layer_norm/batchnorm/subSubNtransformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Atransformer/layer_9/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2= ;transformer/layer_9/self_attention_layer_norm/batchnorm/subĖ =transformer/layer_9/self_attention_layer_norm/batchnorm/add_1AddV2Atransformer/layer_9/self_attention_layer_norm/batchnorm/mul_1:z:0?transformer/layer_9/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer/layer_9/self_attention_layer_norm/batchnorm/add_1‡ =transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpFtransformer_layer_9_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02? =transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpŲ .transformer/layer_9/intermediate/einsum/EinsumEinsumAtransformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0Etransformer/layer_9/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd20 .transformer/layer_9/intermediate/einsum/Einsumä 3transformer/layer_9/intermediate/add/ReadVariableOpReadVariableOpabd2* (transformer/layer_9/output/einsum/EinsumŅ -transformer/layer_9/output/add/ReadVariableOpReadVariableOp6transformer_layer_9_output_add_readvariableop_resource* _output_shapes :€* dtype02/ -transformer/layer_9/output/add/ReadVariableOpó transformer/layer_9/output/addAddV21transformer/layer_9/output/einsum/Einsum:output:05transformer/layer_9/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_9/output/add” ,transformer/layer_9/dropout_29/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2. ,transformer/layer_9/dropout_29/dropout/Constś *transformer/layer_9/dropout_29/dropout/MulMul"transformer/layer_9/output/add:z:05transformer/layer_9/dropout_29/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2, *transformer/layer_9/dropout_29/dropout/Mul® ,transformer/layer_9/dropout_29/dropout/ShapeShape"transformer/layer_9/output/add:z:0* T0* _output_shapes :2. ,transformer/layer_9/dropout_29/dropout/ShapeŸ Ctransformer/layer_9/dropout_29/dropout/random_uniform/RandomUniform RandomUniform5transformer/layer_9/dropout_29/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02E Ctransformer/layer_9/dropout_29/dropout/random_uniform/RandomUniform³ 5transformer/layer_9/dropout_29/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=27 5transformer/layer_9/dropout_29/dropout/GreaterEqual/yČ 3transformer/layer_9/dropout_29/dropout/GreaterEqual GreaterEqualLtransformer/layer_9/dropout_29/dropout/random_uniform/RandomUniform:output:0>transformer/layer_9/dropout_29/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_9/dropout_29/dropout/GreaterEqualź +transformer/layer_9/dropout_29/dropout/CastCast7transformer/layer_9/dropout_29/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_9/dropout_29/dropout/Cast„ ,transformer/layer_9/dropout_29/dropout/Mul_1Mul.transformer/layer_9/dropout_29/dropout/Mul:z:0/transformer/layer_9/dropout_29/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_9/dropout_29/dropout/Mul_1ō transformer/layer_9/add_1AddV20transformer/layer_9/dropout_29/dropout/Mul_1:z:0Atransformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_9/add_1Ö Dtransformer/layer_9/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_9/output_layer_norm/moments/mean/reduction_indices® 2transformer/layer_9/output_layer_norm/moments/meanMeantransformer/layer_9/add_1:z:0Mtransformer/layer_9/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(24 2transformer/layer_9/output_layer_norm/moments/mean„ :transformer/layer_9/output_layer_norm/moments/StopGradient StopGradient;transformer/layer_9/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2< :transformer/layer_9/output_layer_norm/moments/StopGradient» ?transformer/layer_9/output_layer_norm/moments/SquaredDifferenceSquaredDifferencetransformer/layer_9/add_1:z:0Ctransformer/layer_9/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer/layer_9/output_layer_norm/moments/SquaredDifferenceŽ Htransformer/layer_9/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2J Htransformer/layer_9/output_layer_norm/moments/variance/reduction_indicesą 6transformer/layer_9/output_layer_norm/moments/varianceMeanCtransformer/layer_9/output_layer_norm/moments/SquaredDifference:z:0Qtransformer/layer_9/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(28 6transformer/layer_9/output_layer_norm/moments/variance³ 5transformer/layer_9/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+27 5transformer/layer_9/output_layer_norm/batchnorm/add/y³ 3transformer/layer_9/output_layer_norm/batchnorm/addAddV2?transformer/layer_9/output_layer_norm/moments/variance:output:0>transformer/layer_9/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’25 3transformer/layer_9/output_layer_norm/batchnorm/addļ 5transformer/layer_9/output_layer_norm/batchnorm/RsqrtRsqrt7transformer/layer_9/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’27 5transformer/layer_9/output_layer_norm/batchnorm/Rsqrt‘ Btransformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpKtransformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02D Btransformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpø 3transformer/layer_9/output_layer_norm/batchnorm/mulMul9transformer/layer_9/output_layer_norm/batchnorm/Rsqrt:y:0Jtransformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_9/output_layer_norm/batchnorm/mul 5transformer/layer_9/output_layer_norm/batchnorm/mul_1Multransformer/layer_9/add_1:z:07transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_9/output_layer_norm/batchnorm/mul_1« 5transformer/layer_9/output_layer_norm/batchnorm/mul_2Mul;transformer/layer_9/output_layer_norm/moments/mean:output:07transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_9/output_layer_norm/batchnorm/mul_2… >transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOpReadVariableOpGtransformer_layer_9_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02@ >transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp“ 3transformer/layer_9/output_layer_norm/batchnorm/subSubFtransformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp:value:09transformer/layer_9/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer/layer_9/output_layer_norm/batchnorm/sub« 5transformer/layer_9/output_layer_norm/batchnorm/add_1AddV29transformer/layer_9/output_layer_norm/batchnorm/mul_1:z:07transformer/layer_9/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€27 5transformer/layer_9/output_layer_norm/batchnorm/add_1„ Ftransformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpOtransformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02H Ftransformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpš 7transformer/layer_10/self_attention/query/einsum/EinsumEinsum9transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0Ntransformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde29 7transformer/layer_10/self_attention/query/einsum/Einsum‚ abde27 5transformer/layer_10/self_attention/key/einsum/Einsumü :transformer/layer_10/self_attention/key/add/ReadVariableOpReadVariableOpCtransformer_layer_10_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02< :transformer/layer_10/self_attention/key/add/ReadVariableOpŖ +transformer/layer_10/self_attention/key/addAddV2>transformer/layer_10/self_attention/key/einsum/Einsum:output:0Btransformer/layer_10/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2- +transformer/layer_10/self_attention/key/add„ Ftransformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpOtransformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02H Ftransformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpš 7transformer/layer_10/self_attention/value/einsum/EinsumEinsum9transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0Ntransformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde29 7transformer/layer_10/self_attention/value/einsum/Einsum‚ acbe23 1transformer/layer_10/self_attention/einsum/Einsum› )transformer/layer_10/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2+ )transformer/layer_10/self_attention/Mul/y• 'transformer/layer_10/self_attention/MulMul:transformer/layer_10/self_attention/einsum/Einsum:output:02transformer/layer_10/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'transformer/layer_10/self_attention/MulÖ Dtransformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimŹ @transformer/layer_10/self_attention/masked_softmax_10/ExpandDims ExpandDimsself_attention_mask/mul:z:0Mtransformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2B @transformer/layer_10/self_attention/masked_softmax_10/ExpandDimsæ ;transformer/layer_10/self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2= ;transformer/layer_10/self_attention/masked_softmax_10/sub/xŚ 9transformer/layer_10/self_attention/masked_softmax_10/subSubDtransformer/layer_10/self_attention/masked_softmax_10/sub/x:output:0Itransformer/layer_10/self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_10/self_attention/masked_softmax_10/subæ ;transformer/layer_10/self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2= ;transformer/layer_10/self_attention/masked_softmax_10/mul/yĪ 9transformer/layer_10/self_attention/masked_softmax_10/mulMul=transformer/layer_10/self_attention/masked_softmax_10/sub:z:0Dtransformer/layer_10/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_10/self_attention/masked_softmax_10/mul· 9transformer/layer_10/self_attention/masked_softmax_10/addAddV2+transformer/layer_10/self_attention/Mul:z:0=transformer/layer_10/self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_10/self_attention/masked_softmax_10/add” =transformer/layer_10/self_attention/masked_softmax_10/SoftmaxSoftmax=transformer/layer_10/self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_10/self_attention/masked_softmax_10/SoftmaxĮ transformer/layer_10/self_attention/dropout_30/dropout/Mul:z:0?transformer/layer_10/self_attention/dropout_30/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2> abcd25 3transformer/layer_10/self_attention/einsum_1/EinsumĘ Qtransformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02S Qtransformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ Btransformer/layer_10/self_attention/attention_output/einsum/EinsumEinsumabe2D Btransformer/layer_10/self_attention/attention_output/einsum/Einsum  Gtransformer/layer_10/self_attention/attention_output/add/ReadVariableOpReadVariableOpPtransformer_layer_10_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_10/self_attention/attention_output/add/ReadVariableOpŪ 8transformer/layer_10/self_attention/attention_output/addAddV2Ktransformer/layer_10/self_attention/attention_output/einsum/Einsum:output:0Otransformer/layer_10/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer/layer_10/self_attention/attention_output/add£ -transformer/layer_10/dropout_31/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2/ -transformer/layer_10/dropout_31/dropout/Const— +transformer/layer_10/dropout_31/dropout/MulMultransformer/layer_10/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/add/y× transformer/layer_10/self_attention_layer_norm/batchnorm/RsqrtRsqrt@transformer/layer_10/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/Rsqrt¬ Ktransformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpTtransformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02M Ktransformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpÜ transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_10/add:z:0@transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Ļ >transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2MulDtransformer/layer_10/self_attention_layer_norm/moments/mean:output:0@transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2  Gtransformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpPtransformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpŲ transformer/layer_10/self_attention_layer_norm/batchnorm/add_1AddV2Btransformer/layer_10/self_attention_layer_norm/batchnorm/mul_1:z:0@transformer/layer_10/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_10/self_attention_layer_norm/batchnorm/add_1Š >transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpGtransformer_layer_10_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02@ >transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpÜ /transformer/layer_10/intermediate/einsum/EinsumEinsumBtransformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0Ftransformer/layer_10/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd21 /transformer/layer_10/intermediate/einsum/Einsumē 4transformer/layer_10/intermediate/add/ReadVariableOpReadVariableOp=transformer_layer_10_intermediate_add_readvariableop_resource* _output_shapes :€* dtype026 4transformer/layer_10/intermediate/add/ReadVariableOp %transformer/layer_10/intermediate/addAddV28transformer/layer_10/intermediate/einsum/Einsum:output:0abd2+ )transformer/layer_10/output/einsum/EinsumÕ .transformer/layer_10/output/add/ReadVariableOpReadVariableOp7transformer_layer_10_output_add_readvariableop_resource* _output_shapes :€* dtype020 .transformer/layer_10/output/add/ReadVariableOp÷ transformer/layer_10/output/addAddV22transformer/layer_10/output/einsum/Einsum:output:06transformer/layer_10/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! transformer/layer_10/output/add£ -transformer/layer_10/dropout_32/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2/ -transformer/layer_10/dropout_32/dropout/Constž +transformer/layer_10/dropout_32/dropout/MulMul#transformer/layer_10/output/add:z:06transformer/layer_10/dropout_32/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_10/dropout_32/dropout/Mul± -transformer/layer_10/dropout_32/dropout/ShapeShape#transformer/layer_10/output/add:z:0* T0* _output_shapes :2/ -transformer/layer_10/dropout_32/dropout/Shape¢ Dtransformer/layer_10/dropout_32/dropout/random_uniform/RandomUniform RandomUniform6transformer/layer_10/dropout_32/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02F Dtransformer/layer_10/dropout_32/dropout/random_uniform/RandomUniformµ 6transformer/layer_10/dropout_32/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=28 6transformer/layer_10/dropout_32/dropout/GreaterEqual/yĢ 4transformer/layer_10/dropout_32/dropout/GreaterEqual GreaterEqualMtransformer/layer_10/dropout_32/dropout/random_uniform/RandomUniform:output:0?transformer/layer_10/dropout_32/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€26 4transformer/layer_10/dropout_32/dropout/GreaterEqualķ ,transformer/layer_10/dropout_32/dropout/CastCast8transformer/layer_10/dropout_32/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_10/dropout_32/dropout/Castˆ -transformer/layer_10/dropout_32/dropout/Mul_1Mul/transformer/layer_10/dropout_32/dropout/Mul:z:00transformer/layer_10/dropout_32/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer/layer_10/dropout_32/dropout/Mul_1ų transformer/layer_10/add_1AddV21transformer/layer_10/dropout_32/dropout/Mul_1:z:0Btransformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_10/add_1Ų Etransformer/layer_10/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2G Etransformer/layer_10/output_layer_norm/moments/mean/reduction_indices² 3transformer/layer_10/output_layer_norm/moments/meanMeantransformer/layer_10/add_1:z:0Ntransformer/layer_10/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(25 3transformer/layer_10/output_layer_norm/moments/mean‡ ;transformer/layer_10/output_layer_norm/moments/StopGradient StopGradientabde29 7transformer/layer_11/self_attention/query/einsum/Einsum‚ abde27 5transformer/layer_11/self_attention/key/einsum/Einsumü :transformer/layer_11/self_attention/key/add/ReadVariableOpReadVariableOpCtransformer_layer_11_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02< :transformer/layer_11/self_attention/key/add/ReadVariableOpŖ +transformer/layer_11/self_attention/key/addAddV2>transformer/layer_11/self_attention/key/einsum/Einsum:output:0Btransformer/layer_11/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2- +transformer/layer_11/self_attention/key/add„ Ftransformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpOtransformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02H Ftransformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpń 7transformer/layer_11/self_attention/value/einsum/EinsumEinsum:transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0Ntransformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde29 7transformer/layer_11/self_attention/value/einsum/Einsum‚ acbe23 1transformer/layer_11/self_attention/einsum/Einsum› )transformer/layer_11/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2+ )transformer/layer_11/self_attention/Mul/y• 'transformer/layer_11/self_attention/MulMul:transformer/layer_11/self_attention/einsum/Einsum:output:02transformer/layer_11/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'transformer/layer_11/self_attention/MulÖ Dtransformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2F Dtransformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimŹ @transformer/layer_11/self_attention/masked_softmax_11/ExpandDims ExpandDimsself_attention_mask/mul:z:0Mtransformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2B @transformer/layer_11/self_attention/masked_softmax_11/ExpandDimsæ ;transformer/layer_11/self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2= ;transformer/layer_11/self_attention/masked_softmax_11/sub/xŚ 9transformer/layer_11/self_attention/masked_softmax_11/subSubDtransformer/layer_11/self_attention/masked_softmax_11/sub/x:output:0Itransformer/layer_11/self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_11/self_attention/masked_softmax_11/subæ ;transformer/layer_11/self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2= ;transformer/layer_11/self_attention/masked_softmax_11/mul/yĪ 9transformer/layer_11/self_attention/masked_softmax_11/mulMul=transformer/layer_11/self_attention/masked_softmax_11/sub:z:0Dtransformer/layer_11/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_11/self_attention/masked_softmax_11/mul· 9transformer/layer_11/self_attention/masked_softmax_11/addAddV2+transformer/layer_11/self_attention/Mul:z:0=transformer/layer_11/self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2; 9transformer/layer_11/self_attention/masked_softmax_11/add” =transformer/layer_11/self_attention/masked_softmax_11/SoftmaxSoftmax=transformer/layer_11/self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2? =transformer/layer_11/self_attention/masked_softmax_11/SoftmaxĮ transformer/layer_11/self_attention/dropout_33/dropout/Mul:z:0?transformer/layer_11/self_attention/dropout_33/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2> abcd25 3transformer/layer_11/self_attention/einsum_1/EinsumĘ Qtransformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02S Qtransformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOp‘ Btransformer/layer_11/self_attention/attention_output/einsum/EinsumEinsumabe2D Btransformer/layer_11/self_attention/attention_output/einsum/Einsum  Gtransformer/layer_11/self_attention/attention_output/add/ReadVariableOpReadVariableOpPtransformer_layer_11_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_11/self_attention/attention_output/add/ReadVariableOpŪ 8transformer/layer_11/self_attention/attention_output/addAddV2Ktransformer/layer_11/self_attention/attention_output/einsum/Einsum:output:0Otransformer/layer_11/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer/layer_11/self_attention/attention_output/add£ -transformer/layer_11/dropout_34/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2/ -transformer/layer_11/dropout_34/dropout/Const— +transformer/layer_11/dropout_34/dropout/MulMultransformer/layer_11/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/add/y× transformer/layer_11/self_attention_layer_norm/batchnorm/RsqrtRsqrt@transformer/layer_11/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/Rsqrt¬ Ktransformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpTtransformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02M Ktransformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpÜ transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Multransformer/layer_11/add:z:0@transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Ļ >transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2MulDtransformer/layer_11/self_attention_layer_norm/moments/mean:output:0@transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2  Gtransformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpPtransformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpŲ transformer/layer_11/self_attention_layer_norm/batchnorm/add_1AddV2Btransformer/layer_11/self_attention_layer_norm/batchnorm/mul_1:z:0@transformer/layer_11/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer/layer_11/self_attention_layer_norm/batchnorm/add_1Š >transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpGtransformer_layer_11_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02@ >transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpÜ /transformer/layer_11/intermediate/einsum/EinsumEinsumBtransformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0Ftransformer/layer_11/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd21 /transformer/layer_11/intermediate/einsum/Einsumē 4transformer/layer_11/intermediate/add/ReadVariableOpReadVariableOp=transformer_layer_11_intermediate_add_readvariableop_resource* _output_shapes :€* dtype026 4transformer/layer_11/intermediate/add/ReadVariableOp %transformer/layer_11/intermediate/addAddV28transformer/layer_11/intermediate/einsum/Einsum:output:0abd2+ )transformer/layer_11/output/einsum/EinsumÕ .transformer/layer_11/output/add/ReadVariableOpReadVariableOp7transformer_layer_11_output_add_readvariableop_resource* _output_shapes :€* dtype020 .transformer/layer_11/output/add/ReadVariableOp÷ transformer/layer_11/output/addAddV22transformer/layer_11/output/einsum/Einsum:output:06transformer/layer_11/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! transformer/layer_11/output/add£ -transformer/layer_11/dropout_35/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2/ -transformer/layer_11/dropout_35/dropout/Constž +transformer/layer_11/dropout_35/dropout/MulMul#transformer/layer_11/output/add:z:06transformer/layer_11/dropout_35/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer/layer_11/dropout_35/dropout/Mul± -transformer/layer_11/dropout_35/dropout/ShapeShape#transformer/layer_11/output/add:z:0* T0* _output_shapes :2/ -transformer/layer_11/dropout_35/dropout/Shape¢ Dtransformer/layer_11/dropout_35/dropout/random_uniform/RandomUniform RandomUniform6transformer/layer_11/dropout_35/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02F Dtransformer/layer_11/dropout_35/dropout/random_uniform/RandomUniformµ 6transformer/layer_11/dropout_35/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=28 6transformer/layer_11/dropout_35/dropout/GreaterEqual/yĢ 4transformer/layer_11/dropout_35/dropout/GreaterEqual GreaterEqualMtransformer/layer_11/dropout_35/dropout/random_uniform/RandomUniform:output:0?transformer/layer_11/dropout_35/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€26 4transformer/layer_11/dropout_35/dropout/GreaterEqualķ ,transformer/layer_11/dropout_35/dropout/CastCast8transformer/layer_11/dropout_35/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer/layer_11/dropout_35/dropout/Castˆ -transformer/layer_11/dropout_35/dropout/Mul_1Mul/transformer/layer_11/dropout_35/dropout/Mul:z:00transformer/layer_11/dropout_35/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer/layer_11/dropout_35/dropout/Mul_1ų transformer/layer_11/add_1AddV21transformer/layer_11/dropout_35/dropout/Mul_1:z:0Btransformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 transformer/layer_11/add_1Ų Etransformer/layer_11/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2G Etransformer/layer_11/output_layer_norm/moments/mean/reduction_indices² 3transformer/layer_11/output_layer_norm/moments/meanMeantransformer/layer_11/add_1:z:0Ntransformer/layer_11/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(25 3transformer/layer_11/output_layer_norm/moments/mean‡ ;transformer/layer_11/output_layer_norm/moments/StopGradient StopGradient  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : īŗ £ N__inference_transformer/layer_3_layer_call_and_return_conditional_losses_26564 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_3/ExpandDims/dimõ *self_attention/masked_softmax_3/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_3/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_3/ExpandDims“ %self_attention/masked_softmax_3/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_3/sub/x‚ #self_attention/masked_softmax_3/subSub.self_attention/masked_softmax_3/sub/x:output:03self_attention/masked_softmax_3/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/sub“ %self_attention/masked_softmax_3/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_3/mul/yö #self_attention/masked_softmax_3/mulMul'self_attention/masked_softmax_3/sub:z:0.self_attention/masked_softmax_3/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/mulą #self_attention/masked_softmax_3/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/addŅ 'self_attention/masked_softmax_3/SoftmaxSoftmax'self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_3/Softmax• &self_attention/dropout_9/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2( &self_attention/dropout_9/dropout/Constƒ $self_attention/dropout_9/dropout/MulMul1self_attention/masked_softmax_3/Softmax:softmax:0/self_attention/dropout_9/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout_9/dropout/Mul± &self_attention/dropout_9/dropout/ShapeShape1self_attention/masked_softmax_3/Softmax:softmax:0* T0* _output_shapes :2( &self_attention/dropout_9/dropout/Shape™ =self_attention/dropout_9/dropout/random_uniform/RandomUniform RandomUniform/self_attention/dropout_9/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02? =self_attention/dropout_9/dropout/random_uniform/RandomUniform§ /self_attention/dropout_9/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=21 /self_attention/dropout_9/dropout/GreaterEqual/y¼ -self_attention/dropout_9/dropout/GreaterEqual GreaterEqualFself_attention/dropout_9/dropout/random_uniform/RandomUniform:output:08self_attention/dropout_9/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2/ -self_attention/dropout_9/dropout/GreaterEqualä %self_attention/dropout_9/dropout/CastCast1self_attention/dropout_9/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_9/dropout/Castų &self_attention/dropout_9/dropout/Mul_1Mul(self_attention/dropout_9/dropout/Mul:z:0)self_attention/dropout_9/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_9/dropout/Mul_1’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_9/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_10/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_10/dropout/ConstĆ dropout_10/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_10/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/dropout/Mul‹ dropout_10/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_10/dropout/Shapeć /dropout_10/dropout/random_uniform/RandomUniform RandomUniform!dropout_10/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_10/dropout/random_uniform/RandomUniform‹ !dropout_10/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_10/dropout/GreaterEqual/yų dropout_10/dropout/GreaterEqual GreaterEqual8dropout_10/dropout/random_uniform/RandomUniform:output:0*dropout_10/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_10/dropout/GreaterEqual® dropout_10/dropout/CastCast#dropout_10/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/dropout/Cast“ dropout_10/dropout/Mul_1Muldropout_10/dropout/Mul:z:0dropout_10/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/dropout/Mul_1{ addAddV2inputs_0dropout_10/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_3/Pow/yž activation_3/PowPowintermediate/add:z:0activation_3/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Powm activation_3/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_3/mul/xž activation_3/mulMulactivation_3/mul/x:output:0activation_3/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul™ activation_3/addAddV2intermediate/add:z:0activation_3/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/addq activation_3/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_3/mul_1/x¤ activation_3/mul_1Mulactivation_3/mul_1/x:output:0activation_3/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_1† activation_3/TanhTanhactivation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Tanhq activation_3/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_3/add_1/x§ activation_3/add_1AddV2activation_3/add_1/x:output:0activation_3/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/add_1q activation_3/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_3/mul_2/x¦ activation_3/mul_2Mulactivation_3/mul_2/x:output:0activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_2 activation_3/mul_3Mulintermediate/add:z:0activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_3/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_11/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_11/dropout/ConstŖ dropout_11/dropout/MulMuloutput/add:z:0!dropout_11/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/dropout/Mulr dropout_11/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_11/dropout/Shapeć /dropout_11/dropout/random_uniform/RandomUniform RandomUniform!dropout_11/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_11/dropout/random_uniform/RandomUniform‹ !dropout_11/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_11/dropout/GreaterEqual/yų dropout_11/dropout/GreaterEqual GreaterEqual8dropout_11/dropout/random_uniform/RandomUniform:output:0*dropout_11/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_11/dropout/GreaterEqual® dropout_11/dropout/CastCast#dropout_11/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/dropout/Cast“ dropout_11/dropout/Mul_1Muldropout_11/dropout/Mul:z:0dropout_11/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/dropout/Mul_1¤ add_1AddV2dropout_11/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : žš £ N__inference_transformer/layer_4_layer_call_and_return_conditional_losses_26982 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_4/ExpandDims/dimõ *self_attention/masked_softmax_4/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_4/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_4/ExpandDims“ %self_attention/masked_softmax_4/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_4/sub/x‚ #self_attention/masked_softmax_4/subSub.self_attention/masked_softmax_4/sub/x:output:03self_attention/masked_softmax_4/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/sub“ %self_attention/masked_softmax_4/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_4/mul/yö #self_attention/masked_softmax_4/mulMul'self_attention/masked_softmax_4/sub:z:0.self_attention/masked_softmax_4/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/mulą #self_attention/masked_softmax_4/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/addŅ 'self_attention/masked_softmax_4/SoftmaxSoftmax'self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_4/SoftmaxÓ "self_attention/dropout_12/IdentityIdentity1self_attention/masked_softmax_4/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_12/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_12/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_13/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/Identity{ addAddV2inputs_0dropout_13/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_4/Pow/yž activation_4/PowPowintermediate/add:z:0activation_4/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Powm activation_4/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_4/mul/xž activation_4/mulMulactivation_4/mul/x:output:0activation_4/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul™ activation_4/addAddV2intermediate/add:z:0activation_4/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/addq activation_4/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_4/mul_1/x¤ activation_4/mul_1Mulactivation_4/mul_1/x:output:0activation_4/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_1† activation_4/TanhTanhactivation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Tanhq activation_4/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_4/add_1/x§ activation_4/add_1AddV2activation_4/add_1/x:output:0activation_4/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/add_1q activation_4/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_4/mul_2/x¦ activation_4/mul_2Mulactivation_4/mul_2/x:output:0activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_2 activation_4/mul_3Mulintermediate/add:z:0activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_4/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_14/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/Identity¤ add_1AddV2dropout_14/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ¹» ¢ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_11771 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_11/ExpandDims/dimų +self_attention/masked_softmax_11/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_11/ExpandDims• &self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_11/sub/x† $self_attention/masked_softmax_11/subSub/self_attention/masked_softmax_11/sub/x:output:04self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/sub• &self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_11/mul/yś $self_attention/masked_softmax_11/mulMul(self_attention/masked_softmax_11/sub:z:0/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/mulć $self_attention/masked_softmax_11/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/addÕ (self_attention/masked_softmax_11/SoftmaxSoftmax(self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_11/Softmax— 'self_attention/dropout_33/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_33/dropout/Const‡ %self_attention/dropout_33/dropout/MulMul2self_attention/masked_softmax_11/Softmax:softmax:00self_attention/dropout_33/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_33/dropout/Mul“ 'self_attention/dropout_33/dropout/ShapeShape2self_attention/masked_softmax_11/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_33/dropout/Shapeœ >self_attention/dropout_33/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_33/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_33/dropout/random_uniform/RandomUniform© 0self_attention/dropout_33/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_33/dropout/GreaterEqual/yĄ .self_attention/dropout_33/dropout/GreaterEqual GreaterEqualGself_attention/dropout_33/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_33/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_33/dropout/GreaterEqualē &self_attention/dropout_33/dropout/CastCast2self_attention/dropout_33/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_33/dropout/Castü 'self_attention/dropout_33/dropout/Mul_1Mul)self_attention/dropout_33/dropout/Mul:z:0*self_attention/dropout_33/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_33/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_33/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_34/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_34/dropout/ConstĆ dropout_34/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_34/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/dropout/Mul‹ dropout_34/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_34/dropout/Shapeć /dropout_34/dropout/random_uniform/RandomUniform RandomUniform!dropout_34/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_34/dropout/random_uniform/RandomUniform‹ !dropout_34/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_34/dropout/GreaterEqual/yų dropout_34/dropout/GreaterEqual GreaterEqual8dropout_34/dropout/random_uniform/RandomUniform:output:0*dropout_34/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_34/dropout/GreaterEqual® dropout_34/dropout/CastCast#dropout_34/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/dropout/Cast“ dropout_34/dropout/Mul_1Muldropout_34/dropout/Mul:z:0dropout_34/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/dropout/Mul_1y addAddV2inputsdropout_34/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_11/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_11/Pow/y” activation_11/PowPowintermediate/add:z:0activation_11/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Powo activation_11/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_11/mul/x¢ activation_11/mulMulactivation_11/mul/x:output:0activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mulœ activation_11/addAddV2intermediate/add:z:0activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/adds activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_11/mul_1/xØ activation_11/mul_1Mulactivation_11/mul_1/x:output:0activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_1‰ activation_11/TanhTanhactivation_11/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Tanhs activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_11/add_1/x« activation_11/add_1AddV2activation_11/add_1/x:output:0activation_11/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/add_1s activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_11/mul_2/xŖ activation_11/mul_2Mulactivation_11/mul_2/x:output:0activation_11/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_2  activation_11/mul_3Mulintermediate/add:z:0activation_11/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_11/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_35/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_35/dropout/ConstŖ dropout_35/dropout/MulMuloutput/add:z:0!dropout_35/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/dropout/Mulr dropout_35/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_35/dropout/Shapeć /dropout_35/dropout/random_uniform/RandomUniform RandomUniform!dropout_35/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_35/dropout/random_uniform/RandomUniform‹ !dropout_35/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_35/dropout/GreaterEqual/yų dropout_35/dropout/GreaterEqual GreaterEqual8dropout_35/dropout/random_uniform/RandomUniform:output:0*dropout_35/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_35/dropout/GreaterEqual® dropout_35/dropout/CastCast#dropout_35/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/dropout/Cast“ dropout_35/dropout/Mul_1Muldropout_35/dropout/Mul:z:0dropout_35/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/dropout/Mul_1¤ add_1AddV2dropout_35/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_1_layer_call_fn_26087 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_84182 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : „ ļ 3__inference_transformer/layer_6_layer_call_fn_27680 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_101482 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ޚ ¤ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_29159 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_11/ExpandDims/dimų +self_attention/masked_softmax_11/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_11/ExpandDims• &self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_11/sub/x† $self_attention/masked_softmax_11/subSub/self_attention/masked_softmax_11/sub/x:output:04self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/sub• &self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_11/mul/yś $self_attention/masked_softmax_11/mulMul(self_attention/masked_softmax_11/sub:z:0/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/mulć $self_attention/masked_softmax_11/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_11/addÕ (self_attention/masked_softmax_11/SoftmaxSoftmax(self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_11/SoftmaxŌ "self_attention/dropout_33/IdentityIdentity2self_attention/masked_softmax_11/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_33/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_33/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_34/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_34/Identity{ addAddV2inputs_0dropout_34/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_11/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_11/Pow/y” activation_11/PowPowintermediate/add:z:0activation_11/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Powo activation_11/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_11/mul/x¢ activation_11/mulMulactivation_11/mul/x:output:0activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mulœ activation_11/addAddV2intermediate/add:z:0activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/adds activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_11/mul_1/xØ activation_11/mul_1Mulactivation_11/mul_1/x:output:0activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_1‰ activation_11/TanhTanhactivation_11/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/Tanhs activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_11/add_1/x« activation_11/add_1AddV2activation_11/add_1/x:output:0activation_11/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/add_1s activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_11/mul_2/xŖ activation_11/mul_2Mulactivation_11/mul_2/x:output:0activation_11/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_2  activation_11/mul_3Mulintermediate/add:z:0activation_11/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_11/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_11/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_35/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_35/Identity¤ add_1AddV2dropout_35/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ē» ¤ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_28741 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_10/ExpandDims/dimų +self_attention/masked_softmax_10/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_10/ExpandDims• &self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_10/sub/x† $self_attention/masked_softmax_10/subSub/self_attention/masked_softmax_10/sub/x:output:04self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/sub• &self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_10/mul/yś $self_attention/masked_softmax_10/mulMul(self_attention/masked_softmax_10/sub:z:0/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/mulć $self_attention/masked_softmax_10/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/addÕ (self_attention/masked_softmax_10/SoftmaxSoftmax(self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_10/Softmax— 'self_attention/dropout_30/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_30/dropout/Const‡ %self_attention/dropout_30/dropout/MulMul2self_attention/masked_softmax_10/Softmax:softmax:00self_attention/dropout_30/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_30/dropout/Mul“ 'self_attention/dropout_30/dropout/ShapeShape2self_attention/masked_softmax_10/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_30/dropout/Shapeœ >self_attention/dropout_30/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_30/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_30/dropout/random_uniform/RandomUniform© 0self_attention/dropout_30/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_30/dropout/GreaterEqual/yĄ .self_attention/dropout_30/dropout/GreaterEqual GreaterEqualGself_attention/dropout_30/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_30/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_30/dropout/GreaterEqualē &self_attention/dropout_30/dropout/CastCast2self_attention/dropout_30/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_30/dropout/Castü 'self_attention/dropout_30/dropout/Mul_1Mul)self_attention/dropout_30/dropout/Mul:z:0*self_attention/dropout_30/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_30/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_30/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_31/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_31/dropout/ConstĆ dropout_31/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_31/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/dropout/Mul‹ dropout_31/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_31/dropout/Shapeć /dropout_31/dropout/random_uniform/RandomUniform RandomUniform!dropout_31/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_31/dropout/random_uniform/RandomUniform‹ !dropout_31/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_31/dropout/GreaterEqual/yų dropout_31/dropout/GreaterEqual GreaterEqual8dropout_31/dropout/random_uniform/RandomUniform:output:0*dropout_31/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_31/dropout/GreaterEqual® dropout_31/dropout/CastCast#dropout_31/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/dropout/Cast“ dropout_31/dropout/Mul_1Muldropout_31/dropout/Mul:z:0dropout_31/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/dropout/Mul_1{ addAddV2inputs_0dropout_31/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_10/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_10/Pow/y” activation_10/PowPowintermediate/add:z:0activation_10/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Powo activation_10/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_10/mul/x¢ activation_10/mulMulactivation_10/mul/x:output:0activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mulœ activation_10/addAddV2intermediate/add:z:0activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/adds activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_10/mul_1/xØ activation_10/mul_1Mulactivation_10/mul_1/x:output:0activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_1‰ activation_10/TanhTanhactivation_10/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Tanhs activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_10/add_1/x« activation_10/add_1AddV2activation_10/add_1/x:output:0activation_10/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/add_1s activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_10/mul_2/xŖ activation_10/mul_2Mulactivation_10/mul_2/x:output:0activation_10/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_2  activation_10/mul_3Mulintermediate/add:z:0activation_10/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_10/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_32/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_32/dropout/ConstŖ dropout_32/dropout/MulMuloutput/add:z:0!dropout_32/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/dropout/Mulr dropout_32/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_32/dropout/Shapeć /dropout_32/dropout/random_uniform/RandomUniform RandomUniform!dropout_32/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_32/dropout/random_uniform/RandomUniform‹ !dropout_32/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_32/dropout/GreaterEqual/yų dropout_32/dropout/GreaterEqual GreaterEqual8dropout_32/dropout/random_uniform/RandomUniform:output:0*dropout_32/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_32/dropout/GreaterEqual® dropout_32/dropout/CastCast#dropout_32/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/dropout/Cast“ dropout_32/dropout/Mul_1Muldropout_32/dropout/Mul:z:0dropout_32/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/dropout/Mul_1¤ add_1AddV2dropout_32/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : † š 4__inference_transformer/layer_11_layer_call_fn_29197 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŅ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_118782 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ė u =__inference_add_layer_call_and_return_conditional_losses_7713 inputs inputs_1 inputs_2 identitye addAddV2inputsinputs_1* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 addj add_1AddV2add:z:0inputs_2* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1k IdentityIdentity add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*v _input_shapese c: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs Œš   M__inference_transformer/layer_3_layer_call_and_return_conditional_losses_9110 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_3/ExpandDims/dimõ *self_attention/masked_softmax_3/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_3/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_3/ExpandDims“ %self_attention/masked_softmax_3/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_3/sub/x‚ #self_attention/masked_softmax_3/subSub.self_attention/masked_softmax_3/sub/x:output:03self_attention/masked_softmax_3/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/sub“ %self_attention/masked_softmax_3/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_3/mul/yö #self_attention/masked_softmax_3/mulMul'self_attention/masked_softmax_3/sub:z:0.self_attention/masked_softmax_3/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/mulą #self_attention/masked_softmax_3/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/addŅ 'self_attention/masked_softmax_3/SoftmaxSoftmax'self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_3/SoftmaxŃ !self_attention/dropout_9/IdentityIdentity1self_attention/masked_softmax_3/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/dropout_9/Identity’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_9/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_10/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/Identityy addAddV2inputsdropout_10/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_3/Pow/yž activation_3/PowPowintermediate/add:z:0activation_3/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Powm activation_3/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_3/mul/xž activation_3/mulMulactivation_3/mul/x:output:0activation_3/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul™ activation_3/addAddV2intermediate/add:z:0activation_3/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/addq activation_3/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_3/mul_1/x¤ activation_3/mul_1Mulactivation_3/mul_1/x:output:0activation_3/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_1† activation_3/TanhTanhactivation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Tanhq activation_3/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_3/add_1/x§ activation_3/add_1AddV2activation_3/add_1/x:output:0activation_3/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/add_1q activation_3/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_3/mul_2/x¦ activation_3/mul_2Mulactivation_3/mul_2/x:output:0activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_2 activation_3/mul_3Mulintermediate/add:z:0activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_3/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_11/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/Identity¤ add_1AddV2dropout_11/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Śp Ż #__inference_signature_wrapper_17809 input_mask input_type_ids input_word_ids unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197 identity identity_1ˆ¢StatefulPartitionedCall† StatefulPartitionedCallStatefulPartitionedCallinput_word_ids input_maskinput_type_idsunknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 unknown_15 unknown_16 unknown_17 unknown_18 unknown_19 unknown_20 unknown_21 unknown_22 unknown_23 unknown_24 unknown_25 unknown_26 unknown_27 unknown_28 unknown_29 unknown_30 unknown_31 unknown_32 unknown_33 unknown_34 unknown_35 unknown_36 unknown_37 unknown_38 unknown_39 unknown_40 unknown_41 unknown_42 unknown_43 unknown_44 unknown_45 unknown_46 unknown_47 unknown_48 unknown_49 unknown_50 unknown_51 unknown_52 unknown_53 unknown_54 unknown_55 unknown_56 unknown_57 unknown_58 unknown_59 unknown_60 unknown_61 unknown_62 unknown_63 unknown_64 unknown_65 unknown_66 unknown_67 unknown_68 unknown_69 unknown_70 unknown_71 unknown_72 unknown_73 unknown_74 unknown_75 unknown_76 unknown_77 unknown_78 unknown_79 unknown_80 unknown_81 unknown_82 unknown_83 unknown_84 unknown_85 unknown_86 unknown_87 unknown_88 unknown_89 unknown_90 unknown_91 unknown_92 unknown_93 unknown_94 unknown_95 unknown_96 unknown_97 unknown_98 unknown_99 unknown_100 unknown_101 unknown_102 unknown_103 unknown_104 unknown_105 unknown_106 unknown_107 unknown_108 unknown_109 unknown_110 unknown_111 unknown_112 unknown_113 unknown_114 unknown_115 unknown_116 unknown_117 unknown_118 unknown_119 unknown_120 unknown_121 unknown_122 unknown_123 unknown_124 unknown_125 unknown_126 unknown_127 unknown_128 unknown_129 unknown_130 unknown_131 unknown_132 unknown_133 unknown_134 unknown_135 unknown_136 unknown_137 unknown_138 unknown_139 unknown_140 unknown_141 unknown_142 unknown_143 unknown_144 unknown_145 unknown_146 unknown_147 unknown_148 unknown_149 unknown_150 unknown_151 unknown_152 unknown_153 unknown_154 unknown_155 unknown_156 unknown_157 unknown_158 unknown_159 unknown_160 unknown_161 unknown_162 unknown_163 unknown_164 unknown_165 unknown_166 unknown_167 unknown_168 unknown_169 unknown_170 unknown_171 unknown_172 unknown_173 unknown_174 unknown_175 unknown_176 unknown_177 unknown_178 unknown_179 unknown_180 unknown_181 unknown_182 unknown_183 unknown_184 unknown_185 unknown_186 unknown_187 unknown_188 unknown_189 unknown_190 unknown_191 unknown_192 unknown_193 unknown_194 unknown_195 unknown_196 unknown_197*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*( f#R! __inference__wrapped_model_76082 StatefulPartitionedCall IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 Identity  Identity_1Identity StatefulPartitionedCall:output:1^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:\X 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ $ _user_specified_name  input_mask:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_type_ids:`\ 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ ( _user_specified_nameinput_word_ids:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : ‘™ £ N__inference_transformer/layer_0_layer_call_and_return_conditional_losses_25738 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¦ ,self_attention/masked_softmax/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2. ,self_attention/masked_softmax/ExpandDims/dimļ (self_attention/masked_softmax/ExpandDims ExpandDimsinputs_15self_attention/masked_softmax/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax/ExpandDims #self_attention/masked_softmax/sub/xConst* _output_shapes :* dtype0* value B *€?2% #self_attention/masked_softmax/sub/xś !self_attention/masked_softmax/subSub,self_attention/masked_softmax/sub/x:output:01self_attention/masked_softmax/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/sub #self_attention/masked_softmax/mul/yConst* _output_shapes :* dtype0* value B *@Ę2% #self_attention/masked_softmax/mul/yī !self_attention/masked_softmax/mulMul%self_attention/masked_softmax/sub:z:0,self_attention/masked_softmax/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/mulŚ !self_attention/masked_softmax/addAddV2self_attention/Mul:z:0%self_attention/masked_softmax/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/masked_softmax/addĢ %self_attention/masked_softmax/SoftmaxSoftmax%self_attention/masked_softmax/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/masked_softmax/SoftmaxĖ self_attention/dropout/IdentityIdentity/self_attention/masked_softmax/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2! self_attention/dropout/Identityż self_attention/einsum_1/EinsumEinsum(self_attention/dropout/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/add dropout_1/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_1/Identityz addAddV2inputs_0dropout_1/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addi activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation/Pow/y˜ activation/PowPowintermediate/add:z:0activation/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Powi activation/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation/mul/x– activation/mulMulactivation/mul/x:output:0activation/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul“ activation/addAddV2intermediate/add:z:0activation/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/addm activation/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation/mul_1/xœ activation/mul_1Mulactivation/mul_1/x:output:0activation/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_1€ activation/TanhTanhactivation/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/Tanhm activation/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation/add_1/xŸ activation/add_1AddV2activation/add_1/x:output:0activation/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/add_1m activation/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation/mul_2/xž activation/mul_2Mulactivation/mul_2/x:output:0activation/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_2— activation/mul_3Mulintermediate/add:z:0activation/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpŻ output/einsum/EinsumEinsumactivation/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add„ dropout_2/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_2/Identity£ add_1AddV2dropout_2/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : •š £ N__inference_transformer/layer_2_layer_call_and_return_conditional_losses_26360 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_2/ExpandDims/dimõ *self_attention/masked_softmax_2/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_2/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_2/ExpandDims“ %self_attention/masked_softmax_2/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_2/sub/x‚ #self_attention/masked_softmax_2/subSub.self_attention/masked_softmax_2/sub/x:output:03self_attention/masked_softmax_2/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/sub“ %self_attention/masked_softmax_2/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_2/mul/yö #self_attention/masked_softmax_2/mulMul'self_attention/masked_softmax_2/sub:z:0.self_attention/masked_softmax_2/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/mulą #self_attention/masked_softmax_2/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/addŅ 'self_attention/masked_softmax_2/SoftmaxSoftmax'self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_2/SoftmaxŃ !self_attention/dropout_6/IdentityIdentity1self_attention/masked_softmax_2/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/dropout_6/Identity’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_6/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/add dropout_7/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/Identityz addAddV2inputs_0dropout_7/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_2/Pow/yž activation_2/PowPowintermediate/add:z:0activation_2/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Powm activation_2/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_2/mul/xž activation_2/mulMulactivation_2/mul/x:output:0activation_2/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul™ activation_2/addAddV2intermediate/add:z:0activation_2/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/addq activation_2/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_2/mul_1/x¤ activation_2/mul_1Mulactivation_2/mul_1/x:output:0activation_2/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_1† activation_2/TanhTanhactivation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Tanhq activation_2/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_2/add_1/x§ activation_2/add_1AddV2activation_2/add_1/x:output:0activation_2/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/add_1q activation_2/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_2/mul_2/x¦ activation_2/mul_2Mulactivation_2/mul_2/x:output:0activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_2 activation_2/mul_3Mulintermediate/add:z:0activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_2/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add„ dropout_8/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/Identity£ add_1AddV2dropout_8/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ü _ A__inference_dropout_layer_call_and_return_conditional_losses_7778 inputs identity_1h IdentityIdentityinputs* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identityw Identity_1IdentityIdentity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1"! identity_1Identity_1:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs ¾ŗ £ N__inference_transformer/layer_2_layer_call_and_return_conditional_losses_26253 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_2/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_2/ExpandDims/dimõ *self_attention/masked_softmax_2/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_2/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_2/ExpandDims“ %self_attention/masked_softmax_2/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_2/sub/x‚ #self_attention/masked_softmax_2/subSub.self_attention/masked_softmax_2/sub/x:output:03self_attention/masked_softmax_2/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/sub“ %self_attention/masked_softmax_2/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_2/mul/yö #self_attention/masked_softmax_2/mulMul'self_attention/masked_softmax_2/sub:z:0.self_attention/masked_softmax_2/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/mulą #self_attention/masked_softmax_2/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_2/addŅ 'self_attention/masked_softmax_2/SoftmaxSoftmax'self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_2/Softmax• &self_attention/dropout_6/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2( &self_attention/dropout_6/dropout/Constƒ $self_attention/dropout_6/dropout/MulMul1self_attention/masked_softmax_2/Softmax:softmax:0/self_attention/dropout_6/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/dropout_6/dropout/Mul± &self_attention/dropout_6/dropout/ShapeShape1self_attention/masked_softmax_2/Softmax:softmax:0* T0* _output_shapes :2( &self_attention/dropout_6/dropout/Shape™ =self_attention/dropout_6/dropout/random_uniform/RandomUniform RandomUniform/self_attention/dropout_6/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02? =self_attention/dropout_6/dropout/random_uniform/RandomUniform§ /self_attention/dropout_6/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=21 /self_attention/dropout_6/dropout/GreaterEqual/y¼ -self_attention/dropout_6/dropout/GreaterEqual GreaterEqualFself_attention/dropout_6/dropout/random_uniform/RandomUniform:output:08self_attention/dropout_6/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2/ -self_attention/dropout_6/dropout/GreaterEqualä %self_attention/dropout_6/dropout/CastCast1self_attention/dropout_6/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_6/dropout/Castų &self_attention/dropout_6/dropout/Mul_1Mul(self_attention/dropout_6/dropout/Mul:z:0)self_attention/dropout_6/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_6/dropout/Mul_1’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_6/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addw dropout_7/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_7/dropout/ConstĄ dropout_7/dropout/MulMul'self_attention/attention_output/add:z:0 dropout_7/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/Mul‰ dropout_7/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_7/dropout/Shapeą .dropout_7/dropout/random_uniform/RandomUniform RandomUniform dropout_7/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_7/dropout/random_uniform/RandomUniform‰ dropout_7/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_7/dropout/GreaterEqual/yō dropout_7/dropout/GreaterEqual GreaterEqual7dropout_7/dropout/random_uniform/RandomUniform:output:0)dropout_7/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/GreaterEqual« dropout_7/dropout/CastCast"dropout_7/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/Cast° dropout_7/dropout/Mul_1Muldropout_7/dropout/Mul:z:0dropout_7/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_7/dropout/Mul_1z addAddV2inputs_0dropout_7/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_2/Pow/yž activation_2/PowPowintermediate/add:z:0activation_2/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Powm activation_2/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_2/mul/xž activation_2/mulMulactivation_2/mul/x:output:0activation_2/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul™ activation_2/addAddV2intermediate/add:z:0activation_2/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/addq activation_2/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_2/mul_1/x¤ activation_2/mul_1Mulactivation_2/mul_1/x:output:0activation_2/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_1† activation_2/TanhTanhactivation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/Tanhq activation_2/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_2/add_1/x§ activation_2/add_1AddV2activation_2/add_1/x:output:0activation_2/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/add_1q activation_2/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_2/mul_2/x¦ activation_2/mul_2Mulactivation_2/mul_2/x:output:0activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_2 activation_2/mul_3Mulintermediate/add:z:0activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_2/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_2/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addw dropout_8/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_8/dropout/Const§ dropout_8/dropout/MulMuloutput/add:z:0 dropout_8/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/Mulp dropout_8/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_8/dropout/Shapeą .dropout_8/dropout/random_uniform/RandomUniform RandomUniform dropout_8/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype020 .dropout_8/dropout/random_uniform/RandomUniform‰ dropout_8/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2" dropout_8/dropout/GreaterEqual/yō dropout_8/dropout/GreaterEqual GreaterEqual7dropout_8/dropout/random_uniform/RandomUniform:output:0)dropout_8/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/GreaterEqual« dropout_8/dropout/CastCast"dropout_8/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/Cast° dropout_8/dropout/Mul_1Muldropout_8/dropout/Mul:z:0dropout_8/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_8/dropout/Mul_1£ add_1AddV2dropout_8/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ųŗ ” N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_10041 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_6/ExpandDims/dimõ *self_attention/masked_softmax_6/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_6/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_6/ExpandDims“ %self_attention/masked_softmax_6/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_6/sub/x‚ #self_attention/masked_softmax_6/subSub.self_attention/masked_softmax_6/sub/x:output:03self_attention/masked_softmax_6/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/sub“ %self_attention/masked_softmax_6/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_6/mul/yö #self_attention/masked_softmax_6/mulMul'self_attention/masked_softmax_6/sub:z:0.self_attention/masked_softmax_6/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/mulą #self_attention/masked_softmax_6/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/addŅ 'self_attention/masked_softmax_6/SoftmaxSoftmax'self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_6/Softmax— 'self_attention/dropout_18/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_18/dropout/Const† %self_attention/dropout_18/dropout/MulMul1self_attention/masked_softmax_6/Softmax:softmax:00self_attention/dropout_18/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_18/dropout/Mul³ 'self_attention/dropout_18/dropout/ShapeShape1self_attention/masked_softmax_6/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_18/dropout/Shapeœ >self_attention/dropout_18/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_18/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_18/dropout/random_uniform/RandomUniform© 0self_attention/dropout_18/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_18/dropout/GreaterEqual/yĄ .self_attention/dropout_18/dropout/GreaterEqual GreaterEqualGself_attention/dropout_18/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_18/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_18/dropout/GreaterEqualē &self_attention/dropout_18/dropout/CastCast2self_attention/dropout_18/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_18/dropout/Castü 'self_attention/dropout_18/dropout/Mul_1Mul)self_attention/dropout_18/dropout/Mul:z:0*self_attention/dropout_18/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_18/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_18/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_19/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_19/dropout/ConstĆ dropout_19/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_19/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/dropout/Mul‹ dropout_19/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_19/dropout/Shapeć /dropout_19/dropout/random_uniform/RandomUniform RandomUniform!dropout_19/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_19/dropout/random_uniform/RandomUniform‹ !dropout_19/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_19/dropout/GreaterEqual/yų dropout_19/dropout/GreaterEqual GreaterEqual8dropout_19/dropout/random_uniform/RandomUniform:output:0*dropout_19/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_19/dropout/GreaterEqual® dropout_19/dropout/CastCast#dropout_19/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/dropout/Cast“ dropout_19/dropout/Mul_1Muldropout_19/dropout/Mul:z:0dropout_19/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/dropout/Mul_1y addAddV2inputsdropout_19/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_6/Pow/yž activation_6/PowPowintermediate/add:z:0activation_6/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Powm activation_6/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_6/mul/xž activation_6/mulMulactivation_6/mul/x:output:0activation_6/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul™ activation_6/addAddV2intermediate/add:z:0activation_6/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/addq activation_6/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_6/mul_1/x¤ activation_6/mul_1Mulactivation_6/mul_1/x:output:0activation_6/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_1† activation_6/TanhTanhactivation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Tanhq activation_6/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_6/add_1/x§ activation_6/add_1AddV2activation_6/add_1/x:output:0activation_6/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/add_1q activation_6/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_6/mul_2/x¦ activation_6/mul_2Mulactivation_6/mul_2/x:output:0activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_2 activation_6/mul_3Mulintermediate/add:z:0activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_6/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_20/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_20/dropout/ConstŖ dropout_20/dropout/MulMuloutput/add:z:0!dropout_20/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/dropout/Mulr dropout_20/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_20/dropout/Shapeć /dropout_20/dropout/random_uniform/RandomUniform RandomUniform!dropout_20/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_20/dropout/random_uniform/RandomUniform‹ !dropout_20/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_20/dropout/GreaterEqual/yų dropout_20/dropout/GreaterEqual GreaterEqual8dropout_20/dropout/random_uniform/RandomUniform:output:0*dropout_20/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_20/dropout/GreaterEqual® dropout_20/dropout/CastCast#dropout_20/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/dropout/Cast“ dropout_20/dropout/Mul_1Muldropout_20/dropout/Mul:z:0dropout_20/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/dropout/Mul_1¤ add_1AddV2dropout_20/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : † š 4__inference_transformer/layer_10_layer_call_fn_28886 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŅ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*X fSRQ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_115322 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_2_layer_call_fn_26436 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_2_layer_call_and_return_conditional_losses_87642 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ›š £ N__inference_transformer/layer_3_layer_call_and_return_conditional_losses_26671 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_3/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_3/ExpandDims/dimõ *self_attention/masked_softmax_3/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_3/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_3/ExpandDims“ %self_attention/masked_softmax_3/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_3/sub/x‚ #self_attention/masked_softmax_3/subSub.self_attention/masked_softmax_3/sub/x:output:03self_attention/masked_softmax_3/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/sub“ %self_attention/masked_softmax_3/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_3/mul/yö #self_attention/masked_softmax_3/mulMul'self_attention/masked_softmax_3/sub:z:0.self_attention/masked_softmax_3/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/mulą #self_attention/masked_softmax_3/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_3/addŅ 'self_attention/masked_softmax_3/SoftmaxSoftmax'self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_3/SoftmaxŃ !self_attention/dropout_9/IdentityIdentity1self_attention/masked_softmax_3/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/dropout_9/Identity’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_9/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_10/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_10/Identity{ addAddV2inputs_0dropout_10/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_3/Pow/yž activation_3/PowPowintermediate/add:z:0activation_3/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Powm activation_3/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_3/mul/xž activation_3/mulMulactivation_3/mul/x:output:0activation_3/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul™ activation_3/addAddV2intermediate/add:z:0activation_3/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/addq activation_3/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_3/mul_1/x¤ activation_3/mul_1Mulactivation_3/mul_1/x:output:0activation_3/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_1† activation_3/TanhTanhactivation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/Tanhq activation_3/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_3/add_1/x§ activation_3/add_1AddV2activation_3/add_1/x:output:0activation_3/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/add_1q activation_3/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_3/mul_2/x¦ activation_3/mul_2Mulactivation_3/mul_2/x:output:0activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_2 activation_3/mul_3Mulintermediate/add:z:0activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_3/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_3/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_11/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_11/Identity¤ add_1AddV2dropout_11/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : š ” N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_10148 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_6/ExpandDims/dimõ *self_attention/masked_softmax_6/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_6/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_6/ExpandDims“ %self_attention/masked_softmax_6/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_6/sub/x‚ #self_attention/masked_softmax_6/subSub.self_attention/masked_softmax_6/sub/x:output:03self_attention/masked_softmax_6/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/sub“ %self_attention/masked_softmax_6/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_6/mul/yö #self_attention/masked_softmax_6/mulMul'self_attention/masked_softmax_6/sub:z:0.self_attention/masked_softmax_6/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/mulą #self_attention/masked_softmax_6/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/addŅ 'self_attention/masked_softmax_6/SoftmaxSoftmax'self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_6/SoftmaxÓ "self_attention/dropout_18/IdentityIdentity1self_attention/masked_softmax_6/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_18/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_18/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_19/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/Identityy addAddV2inputsdropout_19/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_6/Pow/yž activation_6/PowPowintermediate/add:z:0activation_6/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Powm activation_6/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_6/mul/xž activation_6/mulMulactivation_6/mul/x:output:0activation_6/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul™ activation_6/addAddV2intermediate/add:z:0activation_6/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/addq activation_6/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_6/mul_1/x¤ activation_6/mul_1Mulactivation_6/mul_1/x:output:0activation_6/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_1† activation_6/TanhTanhactivation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Tanhq activation_6/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_6/add_1/x§ activation_6/add_1AddV2activation_6/add_1/x:output:0activation_6/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/add_1q activation_6/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_6/mul_2/x¦ activation_6/mul_2Mulactivation_6/mul_2/x:output:0activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_2 activation_6/mul_3Mulintermediate/add:z:0activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_6/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_20/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/Identity¤ add_1AddV2dropout_20/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ƒ ļ 3__inference_transformer/layer_5_layer_call_fn_27369 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŠ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*V fQRO M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_98022 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ß Œ I__inference_type_embeddings_layer_call_and_return_conditional_losses_7693 inputs" matmul_readvariableop_resource identityˆq Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’2 Reshape/shapek ReshapeReshapeinputsReshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2 Reshapei one_hot/on_valueConst* _output_shapes :* dtype0* value B *€?2 one_hot/on_valuek one_hot/off_valueConst* _output_shapes :* dtype0* value B *2 one_hot/off_value` one_hot/depthConst* _output_shapes :* dtype0* value B:2 one_hot/depth¹ one_hotOneHotReshape:output:0one_hot/depth:output:0one_hot/on_value:output:0one_hot/off_value:output:0* T0* TI0*' _output_shapes : ’’’’’’’’’2 one_hotŽ MatMul/ReadVariableOpReadVariableOpmatmul_readvariableop_resource* _output_shapes : €* dtype02 MatMul/ReadVariableOp~ MatMulMatMulone_hot:output:0MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2 MatMulD ShapeShapeinputs* T0* _output_shapes :2 Shapem concat/values_1Const* _output_shapes :* dtype0* valueB :€2 concat/values_1\ concat/axisConst* _output_shapes :* dtype0* value B:2 concat/axisŠ concatConcatV2Shape:output:0concat/values_1:output:0concat/axis:output:0* N* T0* _output_shapes :2 concat„ Reshape_1ReshapeMatMul:product:0concat:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Reshape_1t IdentityIdentityReshape_1:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*3 _input_shapes" : ’’’’’’’’’ ’’’’’’’’’::XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes : †» £ N__inference_transformer/layer_4_layer_call_and_return_conditional_losses_26875 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_4/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_4/ExpandDims/dimõ *self_attention/masked_softmax_4/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_4/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_4/ExpandDims“ %self_attention/masked_softmax_4/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_4/sub/x‚ #self_attention/masked_softmax_4/subSub.self_attention/masked_softmax_4/sub/x:output:03self_attention/masked_softmax_4/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/sub“ %self_attention/masked_softmax_4/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_4/mul/yö #self_attention/masked_softmax_4/mulMul'self_attention/masked_softmax_4/sub:z:0.self_attention/masked_softmax_4/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/mulą #self_attention/masked_softmax_4/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_4/addŅ 'self_attention/masked_softmax_4/SoftmaxSoftmax'self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_4/Softmax— 'self_attention/dropout_12/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_12/dropout/Const† %self_attention/dropout_12/dropout/MulMul1self_attention/masked_softmax_4/Softmax:softmax:00self_attention/dropout_12/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_12/dropout/Mul³ 'self_attention/dropout_12/dropout/ShapeShape1self_attention/masked_softmax_4/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_12/dropout/Shapeœ >self_attention/dropout_12/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_12/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_12/dropout/random_uniform/RandomUniform© 0self_attention/dropout_12/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_12/dropout/GreaterEqual/yĄ .self_attention/dropout_12/dropout/GreaterEqual GreaterEqualGself_attention/dropout_12/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_12/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_12/dropout/GreaterEqualē &self_attention/dropout_12/dropout/CastCast2self_attention/dropout_12/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_12/dropout/Castü 'self_attention/dropout_12/dropout/Mul_1Mul)self_attention/dropout_12/dropout/Mul:z:0*self_attention/dropout_12/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_12/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_12/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_13/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_13/dropout/ConstĆ dropout_13/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_13/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/dropout/Mul‹ dropout_13/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_13/dropout/Shapeć /dropout_13/dropout/random_uniform/RandomUniform RandomUniform!dropout_13/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_13/dropout/random_uniform/RandomUniform‹ !dropout_13/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_13/dropout/GreaterEqual/yų dropout_13/dropout/GreaterEqual GreaterEqual8dropout_13/dropout/random_uniform/RandomUniform:output:0*dropout_13/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_13/dropout/GreaterEqual® dropout_13/dropout/CastCast#dropout_13/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/dropout/Cast“ dropout_13/dropout/Mul_1Muldropout_13/dropout/Mul:z:0dropout_13/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_13/dropout/Mul_1{ addAddV2inputs_0dropout_13/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_4/Pow/yž activation_4/PowPowintermediate/add:z:0activation_4/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Powm activation_4/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_4/mul/xž activation_4/mulMulactivation_4/mul/x:output:0activation_4/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul™ activation_4/addAddV2intermediate/add:z:0activation_4/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/addq activation_4/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_4/mul_1/x¤ activation_4/mul_1Mulactivation_4/mul_1/x:output:0activation_4/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_1† activation_4/TanhTanhactivation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/Tanhq activation_4/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_4/add_1/x§ activation_4/add_1AddV2activation_4/add_1/x:output:0activation_4/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/add_1q activation_4/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_4/mul_2/x¦ activation_4/mul_2Mulactivation_4/mul_2/x:output:0activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_2 activation_4/mul_3Mulintermediate/add:z:0activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_4/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_4/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_14/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_14/dropout/ConstŖ dropout_14/dropout/MulMuloutput/add:z:0!dropout_14/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/dropout/Mulr dropout_14/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_14/dropout/Shapeć /dropout_14/dropout/random_uniform/RandomUniform RandomUniform!dropout_14/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_14/dropout/random_uniform/RandomUniform‹ !dropout_14/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_14/dropout/GreaterEqual/yų dropout_14/dropout/GreaterEqual GreaterEqual8dropout_14/dropout/random_uniform/RandomUniform:output:0*dropout_14/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_14/dropout/GreaterEqual® dropout_14/dropout/CastCast#dropout_14/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/dropout/Cast“ dropout_14/dropout/Mul_1Muldropout_14/dropout/Mul:z:0dropout_14/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_14/dropout/Mul_1¤ add_1AddV2dropout_14/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Å Į P__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_25424 inputs) %batchnorm_mul_readvariableop_resource% !batchnorm_readvariableop_resource identityˆŠ moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2 moments/mean/reduction_indices„ moments/meanMeaninputs'moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 moments/mean’ moments/StopGradient StopGradientmoments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 moments/StopGradient² moments/SquaredDifferenceSquaredDifferenceinputsmoments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 moments/SquaredDifference’ "moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2$ "moments/variance/reduction_indicesČ moments/varianceMeanmoments/SquaredDifference:z:0+moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 moments/varianceg batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2 batchnorm/add/y› batchnorm/addAddV2moments/variance:output:0batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 batchnorm/add} batchnorm/RsqrtRsqrtbatchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 batchnorm/RsqrtŸ batchnorm/mul/ReadVariableOpReadVariableOp%batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02 batchnorm/mul/ReadVariableOp  batchnorm/mulMulbatchnorm/Rsqrt:y:0$batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/mul„ batchnorm/mul_1Mulinputsbatchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/mul_1“ batchnorm/mul_2Mulmoments/mean:output:0batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/mul_2“ batchnorm/ReadVariableOpReadVariableOp!batchnorm_readvariableop_resource* _output_shapes :€* dtype02 batchnorm/ReadVariableOpœ batchnorm/subSub batchnorm/ReadVariableOp:value:0batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/sub“ batchnorm/add_1AddV2batchnorm/mul_1:z:0batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/add_1u IdentityIdentitybatchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*< _input_shapes+ ): ’’’’’’’’’ ’’’’’’’’’€:::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes ::  _output_shapes : †» £ N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_27497 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_6/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_6/ExpandDims/dimõ *self_attention/masked_softmax_6/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_6/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_6/ExpandDims“ %self_attention/masked_softmax_6/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_6/sub/x‚ #self_attention/masked_softmax_6/subSub.self_attention/masked_softmax_6/sub/x:output:03self_attention/masked_softmax_6/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/sub“ %self_attention/masked_softmax_6/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_6/mul/yö #self_attention/masked_softmax_6/mulMul'self_attention/masked_softmax_6/sub:z:0.self_attention/masked_softmax_6/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/mulą #self_attention/masked_softmax_6/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_6/addŅ 'self_attention/masked_softmax_6/SoftmaxSoftmax'self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_6/Softmax— 'self_attention/dropout_18/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_18/dropout/Const† %self_attention/dropout_18/dropout/MulMul1self_attention/masked_softmax_6/Softmax:softmax:00self_attention/dropout_18/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_18/dropout/Mul³ 'self_attention/dropout_18/dropout/ShapeShape1self_attention/masked_softmax_6/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_18/dropout/Shapeœ >self_attention/dropout_18/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_18/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_18/dropout/random_uniform/RandomUniform© 0self_attention/dropout_18/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_18/dropout/GreaterEqual/yĄ .self_attention/dropout_18/dropout/GreaterEqual GreaterEqualGself_attention/dropout_18/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_18/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_18/dropout/GreaterEqualē &self_attention/dropout_18/dropout/CastCast2self_attention/dropout_18/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_18/dropout/Castü 'self_attention/dropout_18/dropout/Mul_1Mul)self_attention/dropout_18/dropout/Mul:z:0*self_attention/dropout_18/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_18/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_18/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_19/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_19/dropout/ConstĆ dropout_19/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_19/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/dropout/Mul‹ dropout_19/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_19/dropout/Shapeć /dropout_19/dropout/random_uniform/RandomUniform RandomUniform!dropout_19/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_19/dropout/random_uniform/RandomUniform‹ !dropout_19/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_19/dropout/GreaterEqual/yų dropout_19/dropout/GreaterEqual GreaterEqual8dropout_19/dropout/random_uniform/RandomUniform:output:0*dropout_19/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_19/dropout/GreaterEqual® dropout_19/dropout/CastCast#dropout_19/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/dropout/Cast“ dropout_19/dropout/Mul_1Muldropout_19/dropout/Mul:z:0dropout_19/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_19/dropout/Mul_1{ addAddV2inputs_0dropout_19/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_6/Pow/yž activation_6/PowPowintermediate/add:z:0activation_6/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Powm activation_6/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_6/mul/xž activation_6/mulMulactivation_6/mul/x:output:0activation_6/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul™ activation_6/addAddV2intermediate/add:z:0activation_6/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/addq activation_6/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_6/mul_1/x¤ activation_6/mul_1Mulactivation_6/mul_1/x:output:0activation_6/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_1† activation_6/TanhTanhactivation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/Tanhq activation_6/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_6/add_1/x§ activation_6/add_1AddV2activation_6/add_1/x:output:0activation_6/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/add_1q activation_6/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_6/mul_2/x¦ activation_6/mul_2Mulactivation_6/mul_2/x:output:0activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_2 activation_6/mul_3Mulintermediate/add:z:0activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_6/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_6/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_20/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_20/dropout/ConstŖ dropout_20/dropout/MulMuloutput/add:z:0!dropout_20/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/dropout/Mulr dropout_20/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_20/dropout/Shapeć /dropout_20/dropout/random_uniform/RandomUniform RandomUniform!dropout_20/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_20/dropout/random_uniform/RandomUniform‹ !dropout_20/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_20/dropout/GreaterEqual/yų dropout_20/dropout/GreaterEqual GreaterEqual8dropout_20/dropout/random_uniform/RandomUniform:output:0*dropout_20/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_20/dropout/GreaterEqual® dropout_20/dropout/CastCast#dropout_20/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/dropout/Cast“ dropout_20/dropout/Mul_1Muldropout_20/dropout/Mul:z:0dropout_20/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_20/dropout/Mul_1¤ add_1AddV2dropout_20/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ޚ ¤ O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_28848 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/Mul¬ /self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :21 /self_attention/masked_softmax_10/ExpandDims/dimų +self_attention/masked_softmax_10/ExpandDims ExpandDimsinputs_18self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2- +self_attention/masked_softmax_10/ExpandDims• &self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2( &self_attention/masked_softmax_10/sub/x† $self_attention/masked_softmax_10/subSub/self_attention/masked_softmax_10/sub/x:output:04self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/sub• &self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2( &self_attention/masked_softmax_10/mul/yś $self_attention/masked_softmax_10/mulMul(self_attention/masked_softmax_10/sub:z:0/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/mulć $self_attention/masked_softmax_10/addAddV2self_attention/Mul:z:0(self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2& $self_attention/masked_softmax_10/addÕ (self_attention/masked_softmax_10/SoftmaxSoftmax(self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2* (self_attention/masked_softmax_10/SoftmaxŌ "self_attention/dropout_30/IdentityIdentity2self_attention/masked_softmax_10/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_30/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_30/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_31/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_31/Identity{ addAddV2inputs_0dropout_31/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addo activation_10/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_10/Pow/y” activation_10/PowPowintermediate/add:z:0activation_10/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Powo activation_10/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_10/mul/x¢ activation_10/mulMulactivation_10/mul/x:output:0activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mulœ activation_10/addAddV2intermediate/add:z:0activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/adds activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_10/mul_1/xØ activation_10/mul_1Mulactivation_10/mul_1/x:output:0activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_1‰ activation_10/TanhTanhactivation_10/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/Tanhs activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_10/add_1/x« activation_10/add_1AddV2activation_10/add_1/x:output:0activation_10/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/add_1s activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_10/mul_2/xŖ activation_10/mul_2Mulactivation_10/mul_2/x:output:0activation_10/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_2  activation_10/mul_3Mulintermediate/add:z:0activation_10/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_10/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpą output/einsum/EinsumEinsumactivation_10/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_32/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_32/Identity¤ add_1AddV2dropout_32/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ä Ą O__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_7745 inputs) %batchnorm_mul_readvariableop_resource% !batchnorm_readvariableop_resource identityˆŠ moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2 moments/mean/reduction_indices„ moments/meanMeaninputs'moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 moments/mean’ moments/StopGradient StopGradientmoments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 moments/StopGradient² moments/SquaredDifferenceSquaredDifferenceinputsmoments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 moments/SquaredDifference’ "moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2$ "moments/variance/reduction_indicesČ moments/varianceMeanmoments/SquaredDifference:z:0+moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 moments/varianceg batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2 batchnorm/add/y› batchnorm/addAddV2moments/variance:output:0batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 batchnorm/add} batchnorm/RsqrtRsqrtbatchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 batchnorm/RsqrtŸ batchnorm/mul/ReadVariableOpReadVariableOp%batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02 batchnorm/mul/ReadVariableOp  batchnorm/mulMulbatchnorm/Rsqrt:y:0$batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/mul„ batchnorm/mul_1Mulinputsbatchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/mul_1“ batchnorm/mul_2Mulmoments/mean:output:0batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/mul_2“ batchnorm/ReadVariableOpReadVariableOp!batchnorm_readvariableop_resource* _output_shapes :€* dtype02 batchnorm/ReadVariableOpœ batchnorm/subSub batchnorm/ReadVariableOp:value:0batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/sub“ batchnorm/add_1AddV2batchnorm/mul_1:z:0batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 batchnorm/add_1u IdentityIdentitybatchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*< _input_shapes+ ): ’’’’’’’’’ ’’’’’’’’’€:::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:  _output_shapes ::  _output_shapes : ÷ŗ   M__inference_transformer/layer_5_layer_call_and_return_conditional_losses_9695 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_5/ExpandDims/dimõ *self_attention/masked_softmax_5/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_5/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_5/ExpandDims“ %self_attention/masked_softmax_5/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_5/sub/x‚ #self_attention/masked_softmax_5/subSub.self_attention/masked_softmax_5/sub/x:output:03self_attention/masked_softmax_5/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/sub“ %self_attention/masked_softmax_5/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_5/mul/yö #self_attention/masked_softmax_5/mulMul'self_attention/masked_softmax_5/sub:z:0.self_attention/masked_softmax_5/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/mulą #self_attention/masked_softmax_5/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/addŅ 'self_attention/masked_softmax_5/SoftmaxSoftmax'self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_5/Softmax— 'self_attention/dropout_15/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_15/dropout/Const† %self_attention/dropout_15/dropout/MulMul1self_attention/masked_softmax_5/Softmax:softmax:00self_attention/dropout_15/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_15/dropout/Mul³ 'self_attention/dropout_15/dropout/ShapeShape1self_attention/masked_softmax_5/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_15/dropout/Shapeœ >self_attention/dropout_15/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_15/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_15/dropout/random_uniform/RandomUniform© 0self_attention/dropout_15/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_15/dropout/GreaterEqual/yĄ .self_attention/dropout_15/dropout/GreaterEqual GreaterEqualGself_attention/dropout_15/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_15/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_15/dropout/GreaterEqualē &self_attention/dropout_15/dropout/CastCast2self_attention/dropout_15/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_15/dropout/Castü 'self_attention/dropout_15/dropout/Mul_1Mul)self_attention/dropout_15/dropout/Mul:z:0*self_attention/dropout_15/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_15/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_15/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_16/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_16/dropout/ConstĆ dropout_16/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_16/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/dropout/Mul‹ dropout_16/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_16/dropout/Shapeć /dropout_16/dropout/random_uniform/RandomUniform RandomUniform!dropout_16/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_16/dropout/random_uniform/RandomUniform‹ !dropout_16/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_16/dropout/GreaterEqual/yų dropout_16/dropout/GreaterEqual GreaterEqual8dropout_16/dropout/random_uniform/RandomUniform:output:0*dropout_16/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_16/dropout/GreaterEqual® dropout_16/dropout/CastCast#dropout_16/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/dropout/Cast“ dropout_16/dropout/Mul_1Muldropout_16/dropout/Mul:z:0dropout_16/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/dropout/Mul_1y addAddV2inputsdropout_16/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_5/Pow/yž activation_5/PowPowintermediate/add:z:0activation_5/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Powm activation_5/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_5/mul/xž activation_5/mulMulactivation_5/mul/x:output:0activation_5/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul™ activation_5/addAddV2intermediate/add:z:0activation_5/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/addq activation_5/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_5/mul_1/x¤ activation_5/mul_1Mulactivation_5/mul_1/x:output:0activation_5/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_1† activation_5/TanhTanhactivation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Tanhq activation_5/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_5/add_1/x§ activation_5/add_1AddV2activation_5/add_1/x:output:0activation_5/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/add_1q activation_5/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_5/mul_2/x¦ activation_5/mul_2Mulactivation_5/mul_2/x:output:0activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_2 activation_5/mul_3Mulintermediate/add:z:0activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_5/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_17/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_17/dropout/ConstŖ dropout_17/dropout/MulMuloutput/add:z:0!dropout_17/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/dropout/Mulr dropout_17/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_17/dropout/Shapeć /dropout_17/dropout/random_uniform/RandomUniform RandomUniform!dropout_17/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_17/dropout/random_uniform/RandomUniform‹ !dropout_17/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_17/dropout/GreaterEqual/yų dropout_17/dropout/GreaterEqual GreaterEqual8dropout_17/dropout/random_uniform/RandomUniform:output:0*dropout_17/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_17/dropout/GreaterEqual® dropout_17/dropout/CastCast#dropout_17/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/dropout/Cast“ dropout_17/dropout/Mul_1Muldropout_17/dropout/Mul:z:0dropout_17/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/dropout/Mul_1¤ add_1AddV2dropout_17/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ų$ w M__inference_self_attention_mask_layer_call_and_return_conditional_losses_7828 inputs inputs_1 identityD ShapeShapeinputs* T0* _output_shapes :2 Shapet strided_slice/stackConst* _output_shapes :* dtype0* value B :2 strided_slice/stackx strided_slice/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_1x strided_slice/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice/stack_2ā strided_slice StridedSliceShape:output:0strided_slice/stack:output:0strided_slice/stack_1:output:0strided_slice/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slicex strided_slice_1/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_1/stack| strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_1| strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_1/stack_2ģ strided_slice_1 StridedSliceShape:output:0strided_slice_1/stack:output:0 strided_slice_1/stack_1:output:0 strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_1J Shape_1Shapeinputs_1* T0* _output_shapes :2 Shape_1x strided_slice_2/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_2/stack| strided_slice_2/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_2/stack_1| strided_slice_2/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_2/stack_2ī strided_slice_2 StridedSliceShape_1:output:0strided_slice_2/stack:output:0 strided_slice_2/stack_1:output:0 strided_slice_2/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_2x strided_slice_3/stackConst* _output_shapes :* dtype0* value B :2 strided_slice_3/stack| strided_slice_3/stack_1Const* _output_shapes :* dtype0* value B :2 strided_slice_3/stack_1| strided_slice_3/stack_2Const* _output_shapes :* dtype0* value B :2 strided_slice_3/stack_2ī strided_slice_3 StridedSliceShape_1:output:0strided_slice_3/stack:output:0 strided_slice_3/stack_1:output:0 strided_slice_3/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask2 strided_slice_3d Reshape/shape/1Const* _output_shapes :* dtype0* value B:2 Reshape/shape/1  Reshape/shapePackstrided_slice:output:0Reshape/shape/1:output:0strided_slice_3:output:0* N* T0* _output_shapes :2 Reshape/shape~ ReshapeReshapeinputs_1Reshape/shape:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 Reshapet CastCastReshape:output:0* DstT0* SrcT0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 Castn ones/mulMulstrided_slice:output:0strided_slice_1:output:0* T0* _output_shapes :2 ones/mul^ ones/mul_1/yConst* _output_shapes :* dtype0* value B:2 ones/mul_1/ye ones/mul_1Mul ones/mul:z:0ones/mul_1/y:output:0* T0* _output_shapes :2 ones/mul_1] ones/Less/yConst* _output_shapes :* dtype0* value B:č2 ones/Less/ye ones/LessLessones/mul_1:z:0ones/Less/y:output:0* T0* _output_shapes :2 ones/Less` ones/packed/2Const* _output_shapes :* dtype0* value B:2 ones/packed/2š ones/packedPackstrided_slice:output:0strided_slice_1:output:0ones/packed/2:output:0* N* T0* _output_shapes :2 ones/packed] ones/ConstConst* _output_shapes :* dtype0* value B *€?2 ones/Const~ onesFillones/packed:output:0ones/Const:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2 onesr mulMul ones:output:0Cast:y:0* T0*= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2 mulq IdentityIdentitymul:z:0* T0*= _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2 Identity" identityIdentity:output:0*P _input_shapes? =: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs †š   M__inference_transformer/layer_1_layer_call_and_return_conditional_losses_8418 inputs inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOpž "self_attention/query/einsum/EinsumEinsuminputs9self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpų self_attention/key/einsum/EinsumEinsuminputs7self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOpž "self_attention/value/einsum/EinsumEinsuminputs9self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_1/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_1/ExpandDims/dimõ *self_attention/masked_softmax_1/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_1/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_1/ExpandDims“ %self_attention/masked_softmax_1/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_1/sub/x‚ #self_attention/masked_softmax_1/subSub.self_attention/masked_softmax_1/sub/x:output:03self_attention/masked_softmax_1/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/sub“ %self_attention/masked_softmax_1/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_1/mul/yö #self_attention/masked_softmax_1/mulMul'self_attention/masked_softmax_1/sub:z:0.self_attention/masked_softmax_1/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/mulą #self_attention/masked_softmax_1/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_1/addŅ 'self_attention/masked_softmax_1/SoftmaxSoftmax'self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_1/SoftmaxŃ !self_attention/dropout_3/IdentityIdentity1self_attention/masked_softmax_1/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2# !self_attention/dropout_3/Identity’ self_attention/einsum_1/EinsumEinsum*self_attention/dropout_3/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/add dropout_4/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_4/Identityx addAddV2inputsdropout_4/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_1/Pow/yž activation_1/PowPowintermediate/add:z:0activation_1/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Powm activation_1/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_1/mul/xž activation_1/mulMulactivation_1/mul/x:output:0activation_1/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul™ activation_1/addAddV2intermediate/add:z:0activation_1/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/addq activation_1/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_1/mul_1/x¤ activation_1/mul_1Mulactivation_1/mul_1/x:output:0activation_1/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_1† activation_1/TanhTanhactivation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/Tanhq activation_1/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_1/add_1/x§ activation_1/add_1AddV2activation_1/add_1/x:output:0activation_1/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/add_1q activation_1/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_1/mul_2/x¦ activation_1/mul_2Mulactivation_1/mul_2/x:output:0activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_2 activation_1/mul_3Mulintermediate/add:z:0activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_1/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_1/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add„ dropout_5/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_5/Identity£ add_1AddV2dropout_5/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs:ea = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : Ā C '__inference_dropout_layer_call_fn_25460 inputs identityŹ PartitionedCallPartitionedCallinputs* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_dropout_layer_call_and_return_conditional_losses_77782 PartitionedCallz IdentityIdentityPartitionedCall:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs †» £ N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_27808 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_7/ExpandDims/dimõ *self_attention/masked_softmax_7/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_7/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_7/ExpandDims“ %self_attention/masked_softmax_7/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_7/sub/x‚ #self_attention/masked_softmax_7/subSub.self_attention/masked_softmax_7/sub/x:output:03self_attention/masked_softmax_7/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/sub“ %self_attention/masked_softmax_7/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_7/mul/yö #self_attention/masked_softmax_7/mulMul'self_attention/masked_softmax_7/sub:z:0.self_attention/masked_softmax_7/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/mulą #self_attention/masked_softmax_7/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/addŅ 'self_attention/masked_softmax_7/SoftmaxSoftmax'self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_7/Softmax— 'self_attention/dropout_21/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_21/dropout/Const† %self_attention/dropout_21/dropout/MulMul1self_attention/masked_softmax_7/Softmax:softmax:00self_attention/dropout_21/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_21/dropout/Mul³ 'self_attention/dropout_21/dropout/ShapeShape1self_attention/masked_softmax_7/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_21/dropout/Shapeœ >self_attention/dropout_21/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_21/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_21/dropout/random_uniform/RandomUniform© 0self_attention/dropout_21/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_21/dropout/GreaterEqual/yĄ .self_attention/dropout_21/dropout/GreaterEqual GreaterEqualGself_attention/dropout_21/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_21/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_21/dropout/GreaterEqualē &self_attention/dropout_21/dropout/CastCast2self_attention/dropout_21/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_21/dropout/Castü 'self_attention/dropout_21/dropout/Mul_1Mul)self_attention/dropout_21/dropout/Mul:z:0*self_attention/dropout_21/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_21/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_21/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_22/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_22/dropout/ConstĆ dropout_22/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_22/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/dropout/Mul‹ dropout_22/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_22/dropout/Shapeć /dropout_22/dropout/random_uniform/RandomUniform RandomUniform!dropout_22/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_22/dropout/random_uniform/RandomUniform‹ !dropout_22/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_22/dropout/GreaterEqual/yų dropout_22/dropout/GreaterEqual GreaterEqual8dropout_22/dropout/random_uniform/RandomUniform:output:0*dropout_22/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_22/dropout/GreaterEqual® dropout_22/dropout/CastCast#dropout_22/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/dropout/Cast“ dropout_22/dropout/Mul_1Muldropout_22/dropout/Mul:z:0dropout_22/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/dropout/Mul_1{ addAddV2inputs_0dropout_22/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_7/Pow/yž activation_7/PowPowintermediate/add:z:0activation_7/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Powm activation_7/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_7/mul/xž activation_7/mulMulactivation_7/mul/x:output:0activation_7/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul™ activation_7/addAddV2intermediate/add:z:0activation_7/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/addq activation_7/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_7/mul_1/x¤ activation_7/mul_1Mulactivation_7/mul_1/x:output:0activation_7/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_1† activation_7/TanhTanhactivation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Tanhq activation_7/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_7/add_1/x§ activation_7/add_1AddV2activation_7/add_1/x:output:0activation_7/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/add_1q activation_7/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_7/mul_2/x¦ activation_7/mul_2Mulactivation_7/mul_2/x:output:0activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_2 activation_7/mul_3Mulintermediate/add:z:0activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_7/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_23/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_23/dropout/ConstŖ dropout_23/dropout/MulMuloutput/add:z:0!dropout_23/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/dropout/Mulr dropout_23/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_23/dropout/Shapeć /dropout_23/dropout/random_uniform/RandomUniform RandomUniform!dropout_23/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_23/dropout/random_uniform/RandomUniform‹ !dropout_23/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_23/dropout/GreaterEqual/yų dropout_23/dropout/GreaterEqual GreaterEqual8dropout_23/dropout/random_uniform/RandomUniform:output:0*dropout_23/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_23/dropout/GreaterEqual® dropout_23/dropout/CastCast#dropout_23/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/dropout/Cast“ dropout_23/dropout/Mul_1Muldropout_23/dropout/Mul:z:0dropout_23/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/dropout/Mul_1¤ add_1AddV2dropout_23/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : „ ļ 3__inference_transformer/layer_9_layer_call_fn_28613 inputs_0 inputs_1 unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14 identityˆ¢StatefulPartitionedCallŃ StatefulPartitionedCallStatefulPartitionedCallinputs_0inputs_1unknown unknown_0 unknown_1 unknown_2 unknown_3 unknown_4 unknown_5 unknown_6 unknown_7 unknown_8 unknown_9 unknown_10 unknown_11 unknown_12 unknown_13 unknown_14* Tin 2* Tout 2* _collective_manager_ids *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€*2 _read_only_resource_inputs  ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_111862 StatefulPartitionedCallœ IdentityIdentity StatefulPartitionedCall:output:0^StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’::::::::::::::::22 StatefulPartitionedCallStatefulPartitionedCall:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : žš £ N__inference_transformer/layer_5_layer_call_and_return_conditional_losses_27293 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_5/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_5/ExpandDims/dimõ *self_attention/masked_softmax_5/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_5/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_5/ExpandDims“ %self_attention/masked_softmax_5/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_5/sub/x‚ #self_attention/masked_softmax_5/subSub.self_attention/masked_softmax_5/sub/x:output:03self_attention/masked_softmax_5/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/sub“ %self_attention/masked_softmax_5/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_5/mul/yö #self_attention/masked_softmax_5/mulMul'self_attention/masked_softmax_5/sub:z:0.self_attention/masked_softmax_5/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/mulą #self_attention/masked_softmax_5/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_5/addŅ 'self_attention/masked_softmax_5/SoftmaxSoftmax'self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_5/SoftmaxÓ "self_attention/dropout_15/IdentityIdentity1self_attention/masked_softmax_5/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_15/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_15/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_16/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_16/Identity{ addAddV2inputs_0dropout_16/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_5/Pow/yž activation_5/PowPowintermediate/add:z:0activation_5/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Powm activation_5/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_5/mul/xž activation_5/mulMulactivation_5/mul/x:output:0activation_5/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul™ activation_5/addAddV2intermediate/add:z:0activation_5/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/addq activation_5/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_5/mul_1/x¤ activation_5/mul_1Mulactivation_5/mul_1/x:output:0activation_5/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_1† activation_5/TanhTanhactivation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/Tanhq activation_5/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_5/add_1/x§ activation_5/add_1AddV2activation_5/add_1/x:output:0activation_5/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/add_1q activation_5/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_5/mul_2/x¦ activation_5/mul_2Mulactivation_5/mul_2/x:output:0activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_2 activation_5/mul_3Mulintermediate/add:z:0activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_5/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_5/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_17/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_17/Identity¤ add_1AddV2dropout_17/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : †» £ N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_28430 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_9/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_9/ExpandDims/dimõ *self_attention/masked_softmax_9/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_9/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_9/ExpandDims“ %self_attention/masked_softmax_9/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_9/sub/x‚ #self_attention/masked_softmax_9/subSub.self_attention/masked_softmax_9/sub/x:output:03self_attention/masked_softmax_9/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/sub“ %self_attention/masked_softmax_9/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_9/mul/yö #self_attention/masked_softmax_9/mulMul'self_attention/masked_softmax_9/sub:z:0.self_attention/masked_softmax_9/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/mulą #self_attention/masked_softmax_9/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_9/addŅ 'self_attention/masked_softmax_9/SoftmaxSoftmax'self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_9/Softmax— 'self_attention/dropout_27/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2) 'self_attention/dropout_27/dropout/Const† %self_attention/dropout_27/dropout/MulMul1self_attention/masked_softmax_9/Softmax:softmax:00self_attention/dropout_27/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2' %self_attention/dropout_27/dropout/Mul³ 'self_attention/dropout_27/dropout/ShapeShape1self_attention/masked_softmax_9/Softmax:softmax:0* T0* _output_shapes :2) 'self_attention/dropout_27/dropout/Shapeœ >self_attention/dropout_27/dropout/random_uniform/RandomUniform RandomUniform0self_attention/dropout_27/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02@ >self_attention/dropout_27/dropout/random_uniform/RandomUniform© 0self_attention/dropout_27/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=22 0self_attention/dropout_27/dropout/GreaterEqual/yĄ .self_attention/dropout_27/dropout/GreaterEqual GreaterEqualGself_attention/dropout_27/dropout/random_uniform/RandomUniform:output:09self_attention/dropout_27/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’20 .self_attention/dropout_27/dropout/GreaterEqualē &self_attention/dropout_27/dropout/CastCast2self_attention/dropout_27/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2( &self_attention/dropout_27/dropout/Castü 'self_attention/dropout_27/dropout/Mul_1Mul)self_attention/dropout_27/dropout/Mul:z:0*self_attention/dropout_27/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/dropout_27/dropout/Mul_1€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_27/dropout/Mul_1:z:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addy dropout_28/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_28/dropout/ConstĆ dropout_28/dropout/MulMul'self_attention/attention_output/add:z:0!dropout_28/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/dropout/Mul‹ dropout_28/dropout/ShapeShape'self_attention/attention_output/add:z:0* T0* _output_shapes :2 dropout_28/dropout/Shapeć /dropout_28/dropout/random_uniform/RandomUniform RandomUniform!dropout_28/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_28/dropout/random_uniform/RandomUniform‹ !dropout_28/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_28/dropout/GreaterEqual/yų dropout_28/dropout/GreaterEqual GreaterEqual8dropout_28/dropout/random_uniform/RandomUniform:output:0*dropout_28/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_28/dropout/GreaterEqual® dropout_28/dropout/CastCast#dropout_28/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/dropout/Cast“ dropout_28/dropout/Mul_1Muldropout_28/dropout/Mul:z:0dropout_28/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_28/dropout/Mul_1{ addAddV2inputs_0dropout_28/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_9/Pow/yž activation_9/PowPowintermediate/add:z:0activation_9/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Powm activation_9/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_9/mul/xž activation_9/mulMulactivation_9/mul/x:output:0activation_9/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul™ activation_9/addAddV2intermediate/add:z:0activation_9/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/addq activation_9/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_9/mul_1/x¤ activation_9/mul_1Mulactivation_9/mul_1/x:output:0activation_9/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_1† activation_9/TanhTanhactivation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/Tanhq activation_9/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_9/add_1/x§ activation_9/add_1AddV2activation_9/add_1/x:output:0activation_9/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/add_1q activation_9/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_9/mul_2/x¦ activation_9/mul_2Mulactivation_9/mul_2/x:output:0activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_2 activation_9/mul_3Mulintermediate/add:z:0activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_9/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_9/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/addy dropout_29/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout_29/dropout/ConstŖ dropout_29/dropout/MulMuloutput/add:z:0!dropout_29/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/dropout/Mulr dropout_29/dropout/ShapeShapeoutput/add:z:0* T0* _output_shapes :2 dropout_29/dropout/Shapeć /dropout_29/dropout/random_uniform/RandomUniform RandomUniform!dropout_29/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype021 /dropout_29/dropout/random_uniform/RandomUniform‹ !dropout_29/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2# !dropout_29/dropout/GreaterEqual/yų dropout_29/dropout/GreaterEqual GreaterEqual8dropout_29/dropout/random_uniform/RandomUniform:output:0*dropout_29/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! dropout_29/dropout/GreaterEqual® dropout_29/dropout/CastCast#dropout_29/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/dropout/Cast“ dropout_29/dropout/Mul_1Muldropout_29/dropout/Mul:z:0dropout_29/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_29/dropout/Mul_1¤ add_1AddV2dropout_29/dropout/Mul_1:z:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : žš £ N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_27915 inputs_0 inputs_1> :self_attention_query_einsum_einsum_readvariableop_resource4 0self_attention_query_add_readvariableop_resource< 8self_attention_key_einsum_einsum_readvariableop_resource2 .self_attention_key_add_readvariableop_resource> :self_attention_value_einsum_einsum_readvariableop_resource4 0self_attention_value_add_readvariableop_resourceI Eself_attention_attention_output_einsum_einsum_readvariableop_resource? ;self_attention_attention_output_add_readvariableop_resourceC ?self_attention_layer_norm_batchnorm_mul_readvariableop_resource? ;self_attention_layer_norm_batchnorm_readvariableop_resource6 2intermediate_einsum_einsum_readvariableop_resource, (intermediate_add_readvariableop_resource0 ,output_einsum_einsum_readvariableop_resource& "output_add_readvariableop_resource; 7output_layer_norm_batchnorm_mul_readvariableop_resource7 3output_layer_norm_batchnorm_readvariableop_resource identityˆę 1self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/query/einsum/Einsum/ReadVariableOp€ "self_attention/query/einsum/EinsumEinsuminputs_09self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/query/einsum/EinsumĆ 'self_attention/query/add/ReadVariableOpReadVariableOp0self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/query/add/ReadVariableOpŽ self_attention/query/addAddV2+self_attention/query/einsum/Einsum:output:0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/query/addą /self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp8self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype021 /self_attention/key/einsum/Einsum/ReadVariableOpś self_attention/key/einsum/EinsumEinsuminputs_07self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2" self_attention/key/einsum/Einsum½ %self_attention/key/add/ReadVariableOpReadVariableOp.self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02' %self_attention/key/add/ReadVariableOpÖ self_attention/key/addAddV2)self_attention/key/einsum/Einsum:output:0-self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/key/addę 1self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOp:self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype023 1self_attention/value/einsum/Einsum/ReadVariableOp€ "self_attention/value/einsum/EinsumEinsuminputs_09self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2$ "self_attention/value/einsum/EinsumĆ 'self_attention/value/add/ReadVariableOpReadVariableOp0self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02) 'self_attention/value/add/ReadVariableOpŽ self_attention/value/addAddV2+self_attention/value/einsum/Einsum:output:0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2 self_attention/value/addō self_attention/einsum/EinsumEinsumself_attention/key/add:z:0self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2 self_attention/einsum/Einsumq self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2 self_attention/Mul/yĮ self_attention/MulMul%self_attention/einsum/Einsum:output:0self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2 self_attention/MulŖ .self_attention/masked_softmax_7/ExpandDims/dimConst* _output_shapes :* dtype0* value B :20 .self_attention/masked_softmax_7/ExpandDims/dimõ *self_attention/masked_softmax_7/ExpandDims ExpandDimsinputs_17self_attention/masked_softmax_7/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2, *self_attention/masked_softmax_7/ExpandDims“ %self_attention/masked_softmax_7/sub/xConst* _output_shapes :* dtype0* value B *€?2' %self_attention/masked_softmax_7/sub/x‚ #self_attention/masked_softmax_7/subSub.self_attention/masked_softmax_7/sub/x:output:03self_attention/masked_softmax_7/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/sub“ %self_attention/masked_softmax_7/mul/yConst* _output_shapes :* dtype0* value B *@Ę2' %self_attention/masked_softmax_7/mul/yö #self_attention/masked_softmax_7/mulMul'self_attention/masked_softmax_7/sub:z:0.self_attention/masked_softmax_7/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/mulą #self_attention/masked_softmax_7/addAddV2self_attention/Mul:z:0'self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2% #self_attention/masked_softmax_7/addŅ 'self_attention/masked_softmax_7/SoftmaxSoftmax'self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2) 'self_attention/masked_softmax_7/SoftmaxÓ "self_attention/dropout_21/IdentityIdentity1self_attention/masked_softmax_7/Softmax:softmax:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2$ "self_attention/dropout_21/Identity€ self_attention/einsum_1/EinsumEinsum+self_attention/dropout_21/Identity:output:0self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2 self_attention/einsum_1/Einsum‡ abe2/ -self_attention/attention_output/einsum/Einsumį 2self_attention/attention_output/add/ReadVariableOpReadVariableOp;self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype024 2self_attention/attention_output/add/ReadVariableOp‡ #self_attention/attention_output/addAddV26self_attention/attention_output/einsum/Einsum:output:0:self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2% #self_attention/attention_output/addŸ dropout_22/IdentityIdentity'self_attention/attention_output/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_22/Identity{ addAddV2inputs_0dropout_22/Identity:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add¾ 8self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2: 8self_attention_layer_norm/moments/mean/reduction_indicesō &self_attention_layer_norm/moments/meanMeanadd:z:0Aself_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2( &self_attention_layer_norm/moments/meaną .self_attention_layer_norm/moments/StopGradient StopGradient/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’20 .self_attention_layer_norm/moments/StopGradient 3self_attention_layer_norm/moments/SquaredDifferenceSquaredDifferenceadd:z:07self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3self_attention_layer_norm/moments/SquaredDifferenceĘ self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/mulÓ )self_attention_layer_norm/batchnorm/mul_1Muladd:z:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_1ū )self_attention_layer_norm/batchnorm/mul_2Mul/self_attention_layer_norm/moments/mean:output:0+self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/mul_2į 2self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOp;self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype024 2self_attention_layer_norm/batchnorm/ReadVariableOp„ 'self_attention_layer_norm/batchnorm/subSub:self_attention_layer_norm/batchnorm/ReadVariableOp:value:0-self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'self_attention_layer_norm/batchnorm/subū )self_attention_layer_norm/batchnorm/add_1AddV2-self_attention_layer_norm/batchnorm/mul_1:z:0+self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )self_attention_layer_norm/batchnorm/add_1Ė )intermediate/einsum/Einsum/ReadVariableOpReadVariableOp2intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02+ )intermediate/einsum/Einsum/ReadVariableOpˆ intermediate/einsum/EinsumEinsum-self_attention_layer_norm/batchnorm/add_1:z:01intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 intermediate/einsum/EinsumØ intermediate/add/ReadVariableOpReadVariableOp(intermediate_add_readvariableop_resource* _output_shapes :€* dtype02! intermediate/add/ReadVariableOp» intermediate/addAddV2#intermediate/einsum/Einsum:output:0'intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 intermediate/addm activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2 activation_7/Pow/yž activation_7/PowPowintermediate/add:z:0activation_7/Pow/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Powm activation_7/mul/xConst* _output_shapes :* dtype0* value B *'7=2 activation_7/mul/xž activation_7/mulMulactivation_7/mul/x:output:0activation_7/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul™ activation_7/addAddV2intermediate/add:z:0activation_7/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/addq activation_7/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2 activation_7/mul_1/x¤ activation_7/mul_1Mulactivation_7/mul_1/x:output:0activation_7/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_1† activation_7/TanhTanhactivation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/Tanhq activation_7/add_1/xConst* _output_shapes :* dtype0* value B *€?2 activation_7/add_1/x§ activation_7/add_1AddV2activation_7/add_1/x:output:0activation_7/Tanh:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/add_1q activation_7/mul_2/xConst* _output_shapes :* dtype0* value B *?2 activation_7/mul_2/x¦ activation_7/mul_2Mulactivation_7/mul_2/x:output:0activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_2 activation_7/mul_3Mulintermediate/add:z:0activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 activation_7/mul_3¹ #output/einsum/Einsum/ReadVariableOpReadVariableOp,output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02% #output/einsum/Einsum/ReadVariableOpß output/einsum/EinsumEinsumactivation_7/mul_3:z:0+output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2 output/einsum/Einsum– output/add/ReadVariableOpReadVariableOp"output_add_readvariableop_resource* _output_shapes :€* dtype02 output/add/ReadVariableOp£ output/addAddV2output/einsum/Einsum:output:0!output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 output/add† dropout_23/IdentityIdentityoutput/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout_23/Identity¤ add_1AddV2dropout_23/Identity:output:0-self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1® 0output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :22 0output_layer_norm/moments/mean/reduction_indicesŽ output_layer_norm/moments/meanMean add_1:z:09output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2 output_layer_norm/moments/meanČ &output_layer_norm/moments/StopGradient StopGradient'output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2( &output_layer_norm/moments/StopGradientė +output_layer_norm/moments/SquaredDifferenceSquaredDifference add_1:z:0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +output_layer_norm/moments/SquaredDifference¶ 4output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :26 4output_layer_norm/moments/variance/reduction_indices "output_layer_norm/moments/varianceMean/output_layer_norm/moments/SquaredDifference:z:0=output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2$ "output_layer_norm/moments/variance‹ !output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2# !output_layer_norm/batchnorm/add/yć output_layer_norm/batchnorm/addAddV2+output_layer_norm/moments/variance:output:0*output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2! output_layer_norm/batchnorm/add³ !output_layer_norm/batchnorm/RsqrtRsqrt#output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2# !output_layer_norm/batchnorm/RsqrtÕ .output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp7output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype020 .output_layer_norm/batchnorm/mul/ReadVariableOpč output_layer_norm/batchnorm/mulMul%output_layer_norm/batchnorm/Rsqrt:y:06output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/mul½ !output_layer_norm/batchnorm/mul_1Mul add_1:z:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_1Ū !output_layer_norm/batchnorm/mul_2Mul'output_layer_norm/moments/mean:output:0#output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/mul_2É *output_layer_norm/batchnorm/ReadVariableOpReadVariableOp3output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02, *output_layer_norm/batchnorm/ReadVariableOpä output_layer_norm/batchnorm/subSub2output_layer_norm/batchnorm/ReadVariableOp:value:0%output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2! output_layer_norm/batchnorm/subŪ !output_layer_norm/batchnorm/add_1AddV2%output_layer_norm/batchnorm/mul_1:z:0#output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2# !output_layer_norm/batchnorm/add_1‡ IdentityIdentity%output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*Ÿ _input_shapes Š: ’’’’’’’’’ ’’’’’’’’’€:' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:gc = _output_shapes+ ):' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes : ՟ Ł1 G__inference_functional_1_layer_call_and_return_conditional_losses_16997 inputs inputs_1 inputs_2 transformer_encoder_16595 transformer_encoder_16597 transformer_encoder_16599 transformer_encoder_16601 transformer_encoder_16603 transformer_encoder_16605 transformer_encoder_16607 transformer_encoder_16609 transformer_encoder_16611 transformer_encoder_16613 transformer_encoder_16615 transformer_encoder_16617 transformer_encoder_16619 transformer_encoder_16621 transformer_encoder_16623 transformer_encoder_16625 transformer_encoder_16627 transformer_encoder_16629 transformer_encoder_16631 transformer_encoder_16633 transformer_encoder_16635 transformer_encoder_16637 transformer_encoder_16639 transformer_encoder_16641 transformer_encoder_16643 transformer_encoder_16645 transformer_encoder_16647 transformer_encoder_16649 transformer_encoder_16651 transformer_encoder_16653 transformer_encoder_16655 transformer_encoder_16657 transformer_encoder_16659 transformer_encoder_16661 transformer_encoder_16663 transformer_encoder_16665 transformer_encoder_16667 transformer_encoder_16669 transformer_encoder_16671 transformer_encoder_16673 transformer_encoder_16675 transformer_encoder_16677 transformer_encoder_16679 transformer_encoder_16681 transformer_encoder_16683 transformer_encoder_16685 transformer_encoder_16687 transformer_encoder_16689 transformer_encoder_16691 transformer_encoder_16693 transformer_encoder_16695 transformer_encoder_16697 transformer_encoder_16699 transformer_encoder_16701 transformer_encoder_16703 transformer_encoder_16705 transformer_encoder_16707 transformer_encoder_16709 transformer_encoder_16711 transformer_encoder_16713 transformer_encoder_16715 transformer_encoder_16717 transformer_encoder_16719 transformer_encoder_16721 transformer_encoder_16723 transformer_encoder_16725 transformer_encoder_16727 transformer_encoder_16729 transformer_encoder_16731 transformer_encoder_16733 transformer_encoder_16735 transformer_encoder_16737 transformer_encoder_16739 transformer_encoder_16741 transformer_encoder_16743 transformer_encoder_16745 transformer_encoder_16747 transformer_encoder_16749 transformer_encoder_16751 transformer_encoder_16753 transformer_encoder_16755 transformer_encoder_16757 transformer_encoder_16759 transformer_encoder_16761 transformer_encoder_16763 transformer_encoder_16765 transformer_encoder_16767 transformer_encoder_16769 transformer_encoder_16771 transformer_encoder_16773 transformer_encoder_16775 transformer_encoder_16777 transformer_encoder_16779 transformer_encoder_16781 transformer_encoder_16783 transformer_encoder_16785 transformer_encoder_16787 transformer_encoder_16789 transformer_encoder_16791 transformer_encoder_16793 transformer_encoder_16795 transformer_encoder_16797 transformer_encoder_16799 transformer_encoder_16801 transformer_encoder_16803 transformer_encoder_16805 transformer_encoder_16807 transformer_encoder_16809 transformer_encoder_16811 transformer_encoder_16813 transformer_encoder_16815 transformer_encoder_16817 transformer_encoder_16819 transformer_encoder_16821 transformer_encoder_16823 transformer_encoder_16825 transformer_encoder_16827 transformer_encoder_16829 transformer_encoder_16831 transformer_encoder_16833 transformer_encoder_16835 transformer_encoder_16837 transformer_encoder_16839 transformer_encoder_16841 transformer_encoder_16843 transformer_encoder_16845 transformer_encoder_16847 transformer_encoder_16849 transformer_encoder_16851 transformer_encoder_16853 transformer_encoder_16855 transformer_encoder_16857 transformer_encoder_16859 transformer_encoder_16861 transformer_encoder_16863 transformer_encoder_16865 transformer_encoder_16867 transformer_encoder_16869 transformer_encoder_16871 transformer_encoder_16873 transformer_encoder_16875 transformer_encoder_16877 transformer_encoder_16879 transformer_encoder_16881 transformer_encoder_16883 transformer_encoder_16885 transformer_encoder_16887 transformer_encoder_16889 transformer_encoder_16891 transformer_encoder_16893 transformer_encoder_16895 transformer_encoder_16897 transformer_encoder_16899 transformer_encoder_16901 transformer_encoder_16903 transformer_encoder_16905 transformer_encoder_16907 transformer_encoder_16909 transformer_encoder_16911 transformer_encoder_16913 transformer_encoder_16915 transformer_encoder_16917 transformer_encoder_16919 transformer_encoder_16921 transformer_encoder_16923 transformer_encoder_16925 transformer_encoder_16927 transformer_encoder_16929 transformer_encoder_16931 transformer_encoder_16933 transformer_encoder_16935 transformer_encoder_16937 transformer_encoder_16939 transformer_encoder_16941 transformer_encoder_16943 transformer_encoder_16945 transformer_encoder_16947 transformer_encoder_16949 transformer_encoder_16951 transformer_encoder_16953 transformer_encoder_16955 transformer_encoder_16957 transformer_encoder_16959 transformer_encoder_16961 transformer_encoder_16963 transformer_encoder_16965 transformer_encoder_16967 transformer_encoder_16969 transformer_encoder_16971 transformer_encoder_16973 transformer_encoder_16975 transformer_encoder_16977 transformer_encoder_16979 transformer_encoder_16981 transformer_encoder_16983 transformer_encoder_16985 transformer_encoder_16987 transformer_encoder_16989 transformer_encoder_16991 identity identity_1ˆ¢+transformer_encoder/StatefulPartitionedCall”1 +transformer_encoder/StatefulPartitionedCallStatefulPartitionedCallinputsinputs_1inputs_2transformer_encoder_16595transformer_encoder_16597transformer_encoder_16599transformer_encoder_16601transformer_encoder_16603transformer_encoder_16605transformer_encoder_16607transformer_encoder_16609transformer_encoder_16611transformer_encoder_16613transformer_encoder_16615transformer_encoder_16617transformer_encoder_16619transformer_encoder_16621transformer_encoder_16623transformer_encoder_16625transformer_encoder_16627transformer_encoder_16629transformer_encoder_16631transformer_encoder_16633transformer_encoder_16635transformer_encoder_16637transformer_encoder_16639transformer_encoder_16641transformer_encoder_16643transformer_encoder_16645transformer_encoder_16647transformer_encoder_16649transformer_encoder_16651transformer_encoder_16653transformer_encoder_16655transformer_encoder_16657transformer_encoder_16659transformer_encoder_16661transformer_encoder_16663transformer_encoder_16665transformer_encoder_16667transformer_encoder_16669transformer_encoder_16671transformer_encoder_16673transformer_encoder_16675transformer_encoder_16677transformer_encoder_16679transformer_encoder_16681transformer_encoder_16683transformer_encoder_16685transformer_encoder_16687transformer_encoder_16689transformer_encoder_16691transformer_encoder_16693transformer_encoder_16695transformer_encoder_16697transformer_encoder_16699transformer_encoder_16701transformer_encoder_16703transformer_encoder_16705transformer_encoder_16707transformer_encoder_16709transformer_encoder_16711transformer_encoder_16713transformer_encoder_16715transformer_encoder_16717transformer_encoder_16719transformer_encoder_16721transformer_encoder_16723transformer_encoder_16725transformer_encoder_16727transformer_encoder_16729transformer_encoder_16731transformer_encoder_16733transformer_encoder_16735transformer_encoder_16737transformer_encoder_16739transformer_encoder_16741transformer_encoder_16743transformer_encoder_16745transformer_encoder_16747transformer_encoder_16749transformer_encoder_16751transformer_encoder_16753transformer_encoder_16755transformer_encoder_16757transformer_encoder_16759transformer_encoder_16761transformer_encoder_16763transformer_encoder_16765transformer_encoder_16767transformer_encoder_16769transformer_encoder_16771transformer_encoder_16773transformer_encoder_16775transformer_encoder_16777transformer_encoder_16779transformer_encoder_16781transformer_encoder_16783transformer_encoder_16785transformer_encoder_16787transformer_encoder_16789transformer_encoder_16791transformer_encoder_16793transformer_encoder_16795transformer_encoder_16797transformer_encoder_16799transformer_encoder_16801transformer_encoder_16803transformer_encoder_16805transformer_encoder_16807transformer_encoder_16809transformer_encoder_16811transformer_encoder_16813transformer_encoder_16815transformer_encoder_16817transformer_encoder_16819transformer_encoder_16821transformer_encoder_16823transformer_encoder_16825transformer_encoder_16827transformer_encoder_16829transformer_encoder_16831transformer_encoder_16833transformer_encoder_16835transformer_encoder_16837transformer_encoder_16839transformer_encoder_16841transformer_encoder_16843transformer_encoder_16845transformer_encoder_16847transformer_encoder_16849transformer_encoder_16851transformer_encoder_16853transformer_encoder_16855transformer_encoder_16857transformer_encoder_16859transformer_encoder_16861transformer_encoder_16863transformer_encoder_16865transformer_encoder_16867transformer_encoder_16869transformer_encoder_16871transformer_encoder_16873transformer_encoder_16875transformer_encoder_16877transformer_encoder_16879transformer_encoder_16881transformer_encoder_16883transformer_encoder_16885transformer_encoder_16887transformer_encoder_16889transformer_encoder_16891transformer_encoder_16893transformer_encoder_16895transformer_encoder_16897transformer_encoder_16899transformer_encoder_16901transformer_encoder_16903transformer_encoder_16905transformer_encoder_16907transformer_encoder_16909transformer_encoder_16911transformer_encoder_16913transformer_encoder_16915transformer_encoder_16917transformer_encoder_16919transformer_encoder_16921transformer_encoder_16923transformer_encoder_16925transformer_encoder_16927transformer_encoder_16929transformer_encoder_16931transformer_encoder_16933transformer_encoder_16935transformer_encoder_16937transformer_encoder_16939transformer_encoder_16941transformer_encoder_16943transformer_encoder_16945transformer_encoder_16947transformer_encoder_16949transformer_encoder_16951transformer_encoder_16953transformer_encoder_16955transformer_encoder_16957transformer_encoder_16959transformer_encoder_16961transformer_encoder_16963transformer_encoder_16965transformer_encoder_16967transformer_encoder_16969transformer_encoder_16971transformer_encoder_16973transformer_encoder_16975transformer_encoder_16977transformer_encoder_16979transformer_encoder_16981transformer_encoder_16983transformer_encoder_16985transformer_encoder_16987transformer_encoder_16989transformer_encoder_16991*Ų TinŠ Ķ2Ź* Tout 2* _collective_manager_ids *I _output_shapes7 5: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’€*¶ _read_only_resource_inputs— ”‘  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉ** config_proto  CPU  GPU2J8*W fRRP N__inference_transformer_encoder_layer_call_and_return_conditional_losses_137392- +transformer_encoder/StatefulPartitionedCall· IdentityIdentity4transformer_encoder/StatefulPartitionedCall:output:1,^transformer_encoder/StatefulPartitionedCall* T0*( _output_shapes : ’’’’’’’’’€2 IdentityČ Identity_1Identity4transformer_encoder/StatefulPartitionedCall:output:0,^transformer_encoder/StatefulPartitionedCall* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::2Z +transformer_encoder/StatefulPartitionedCall+transformer_encoder/StatefulPartitionedCall:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:XT 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ _user_specified_nameinputs:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes : æ• …y __inference__traced_save_29917 file_prefix) %savev2_variable_1_read_readvariableop 9 5savev2_word_embeddings_embeddings_read_readvariableop< 8savev2_position_embedding_embeddings_read_readvariableop9 5savev2_type_embeddings_embeddings_read_readvariableop: 6savev2_embeddings_layer_norm_gamma_read_readvariableop9 5savev2_embeddings_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_0_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_0_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_0_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_0_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_0_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_0_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_0_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_0_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_0_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_0_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_0_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_0_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_0_output_bias_read_readvariableopJ Fsavev2_transformer_layer_0_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_0_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_1_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_1_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_1_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_1_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_1_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_1_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_1_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_1_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_1_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_1_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_1_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_1_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_1_output_bias_read_readvariableopJ Fsavev2_transformer_layer_1_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_1_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_2_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_2_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_2_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_2_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_2_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_2_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_2_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_2_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_2_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_2_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_2_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_2_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_2_output_bias_read_readvariableopJ Fsavev2_transformer_layer_2_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_2_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_3_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_3_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_3_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_3_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_3_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_3_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_3_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_3_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_3_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_3_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_3_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_3_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_3_output_bias_read_readvariableopJ Fsavev2_transformer_layer_3_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_3_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_4_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_4_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_4_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_4_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_4_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_4_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_4_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_4_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_4_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_4_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_4_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_4_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_4_output_bias_read_readvariableopJ Fsavev2_transformer_layer_4_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_4_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_5_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_5_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_5_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_5_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_5_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_5_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_5_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_5_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_5_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_5_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_5_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_5_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_5_output_bias_read_readvariableopJ Fsavev2_transformer_layer_5_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_5_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_6_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_6_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_6_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_6_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_6_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_6_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_6_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_6_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_6_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_6_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_6_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_6_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_6_output_bias_read_readvariableopJ Fsavev2_transformer_layer_6_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_6_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_7_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_7_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_7_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_7_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_7_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_7_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_7_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_7_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_7_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_7_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_7_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_7_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_7_output_bias_read_readvariableopJ Fsavev2_transformer_layer_7_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_7_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_8_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_8_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_8_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_8_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_8_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_8_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_8_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_8_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_8_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_8_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_8_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_8_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_8_output_bias_read_readvariableopJ Fsavev2_transformer_layer_8_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_8_output_layer_norm_beta_read_readvariableopN Jsavev2_transformer_layer_9_self_attention_query_kernel_read_readvariableopL Hsavev2_transformer_layer_9_self_attention_query_bias_read_readvariableopL Hsavev2_transformer_layer_9_self_attention_key_kernel_read_readvariableopJ Fsavev2_transformer_layer_9_self_attention_key_bias_read_readvariableopN Jsavev2_transformer_layer_9_self_attention_value_kernel_read_readvariableopL Hsavev2_transformer_layer_9_self_attention_value_bias_read_readvariableopY Usavev2_transformer_layer_9_self_attention_attention_output_kernel_read_readvariableopW Ssavev2_transformer_layer_9_self_attention_attention_output_bias_read_readvariableopR Nsavev2_transformer_layer_9_self_attention_layer_norm_gamma_read_readvariableopQ Msavev2_transformer_layer_9_self_attention_layer_norm_beta_read_readvariableopF Bsavev2_transformer_layer_9_intermediate_kernel_read_readvariableopD @savev2_transformer_layer_9_intermediate_bias_read_readvariableop@ :savev2_transformer_layer_9_output_bias_read_readvariableopJ Fsavev2_transformer_layer_9_output_layer_norm_gamma_read_readvariableopI Esavev2_transformer_layer_9_output_layer_norm_beta_read_readvariableopO Ksavev2_transformer_layer_10_self_attention_query_kernel_read_readvariableopM Isavev2_transformer_layer_10_self_attention_query_bias_read_readvariableopM Isavev2_transformer_layer_10_self_attention_key_kernel_read_readvariableopK Gsavev2_transformer_layer_10_self_attention_key_bias_read_readvariableopO Ksavev2_transformer_layer_10_self_attention_value_kernel_read_readvariableopM Isavev2_transformer_layer_10_self_attention_value_bias_read_readvariableopZ Vsavev2_transformer_layer_10_self_attention_attention_output_kernel_read_readvariableopX Tsavev2_transformer_layer_10_self_attention_attention_output_bias_read_readvariableopS Osavev2_transformer_layer_10_self_attention_layer_norm_gamma_read_readvariableopR Nsavev2_transformer_layer_10_self_attention_layer_norm_beta_read_readvariableopG Csavev2_transformer_layer_10_intermediate_kernel_read_readvariableopE Asavev2_transformer_layer_10_intermediate_bias_read_readvariableopA =savev2_transformer_layer_10_output_kernel_read_readvariableop? ;savev2_transformer_layer_10_output_bias_read_readvariableopK Gsavev2_transformer_layer_10_output_layer_norm_gamma_read_readvariableopJ Fsavev2_transformer_layer_10_output_layer_norm_beta_read_readvariableopO Ksavev2_transformer_layer_11_self_attention_query_kernel_read_readvariableopM Isavev2_transformer_layer_11_self_attention_query_bias_read_readvariableopM Isavev2_transformer_layer_11_self_attention_key_kernel_read_readvariableopK Gsavev2_transformer_layer_11_self_attention_key_bias_read_readvariableopO Ksavev2_transformer_layer_11_self_attention_value_kernel_read_readvariableopM Isavev2_transformer_layer_11_self_attention_value_bias_read_readvariableopZ Vsavev2_transformer_layer_11_self_attention_attention_output_kernel_read_readvariableopX Tsavev2_transformer_layer_11_self_attention_attention_output_bias_read_readvariableopS Osavev2_transformer_layer_11_self_attention_layer_norm_gamma_read_readvariableopR Nsavev2_transformer_layer_11_self_attention_layer_norm_beta_read_readvariableopG Csavev2_transformer_layer_11_intermediate_kernel_read_readvariableopE Asavev2_transformer_layer_11_intermediate_bias_read_readvariableopA =savev2_transformer_layer_11_output_kernel_read_readvariableop? ;savev2_transformer_layer_11_output_bias_read_readvariableopK Gsavev2_transformer_layer_11_output_layer_norm_gamma_read_readvariableopJ Fsavev2_transformer_layer_11_output_layer_norm_beta_read_readvariableop6 2savev2_pooler_transform_kernel_read_readvariableop4 0savev2_pooler_transform_bias_read_readvariableop savev2_1_const identity_1ˆ¢MergeV2Checkpoints¢SaveV2¢SaveV2_1 StaticRegexFullMatchStaticRegexFullMatch file_prefix" /device:CPU:** _output_shapes :* pattern ^s3://.*2 StaticRegexFullMatchc ConstConst" /device:CPU:** _output_shapes :* dtype0* value B B.part2 Const Const_1Const" /device:CPU:** _output_shapes :* dtype0*< value3B1B+_temp_afa462c202864ab9ba19b279e7d97535/part2 Const_1‹ SelectSelectStaticRegexFullMatch:output:0Const:output:0Const_1:output:0" /device:CPU:** T0* _output_shapes :2 Selectt StringJoin StringJoin file_prefixSelect:output:0" /device:CPU:** N* _output_shapes :2 StringJoinZ num_shardsConst* _output_shapes :* dtype0* value B:2 num_shards ShardedFilename/shardConst" /device:CPU:0* _output_shapes :* dtype0* value B:2 ShardedFilename/shard¦ ShardedFilenameShardedFilenameStringJoin:output:0ShardedFilename/shard:output:0num_shards:output:0" /device:CPU:0* _output_shapes :2 ShardedFilenameēA SaveV2/tensor_namesConst" /device:CPU:0* _output_shapes :Č* dtype0*ų@ valueī@Bė@ČB(do_lower_case/.ATTRIBUTES/VARIABLE_VALUEB&variables/0/.ATTRIBUTES/VARIABLE_VALUEB&variables/1/.ATTRIBUTES/VARIABLE_VALUEB&variables/2/.ATTRIBUTES/VARIABLE_VALUEB&variables/3/.ATTRIBUTES/VARIABLE_VALUEB&variables/4/.ATTRIBUTES/VARIABLE_VALUEB&variables/5/.ATTRIBUTES/VARIABLE_VALUEB&variables/6/.ATTRIBUTES/VARIABLE_VALUEB&variables/7/.ATTRIBUTES/VARIABLE_VALUEB&variables/8/.ATTRIBUTES/VARIABLE_VALUEB&variables/9/.ATTRIBUTES/VARIABLE_VALUEB'variables/10/.ATTRIBUTES/VARIABLE_VALUEB'variables/11/.ATTRIBUTES/VARIABLE_VALUEB'variables/12/.ATTRIBUTES/VARIABLE_VALUEB'variables/13/.ATTRIBUTES/VARIABLE_VALUEB'variables/14/.ATTRIBUTES/VARIABLE_VALUEB'variables/15/.ATTRIBUTES/VARIABLE_VALUEB'variables/16/.ATTRIBUTES/VARIABLE_VALUEB'variables/17/.ATTRIBUTES/VARIABLE_VALUEB'variables/18/.ATTRIBUTES/VARIABLE_VALUEB'variables/19/.ATTRIBUTES/VARIABLE_VALUEB'variables/20/.ATTRIBUTES/VARIABLE_VALUEB'variables/21/.ATTRIBUTES/VARIABLE_VALUEB'variables/22/.ATTRIBUTES/VARIABLE_VALUEB'variables/23/.ATTRIBUTES/VARIABLE_VALUEB'variables/24/.ATTRIBUTES/VARIABLE_VALUEB'variables/25/.ATTRIBUTES/VARIABLE_VALUEB'variables/26/.ATTRIBUTES/VARIABLE_VALUEB'variables/27/.ATTRIBUTES/VARIABLE_VALUEB'variables/28/.ATTRIBUTES/VARIABLE_VALUEB'variables/29/.ATTRIBUTES/VARIABLE_VALUEB'variables/30/.ATTRIBUTES/VARIABLE_VALUEB'variables/31/.ATTRIBUTES/VARIABLE_VALUEB'variables/32/.ATTRIBUTES/VARIABLE_VALUEB'variables/33/.ATTRIBUTES/VARIABLE_VALUEB'variables/34/.ATTRIBUTES/VARIABLE_VALUEB'variables/35/.ATTRIBUTES/VARIABLE_VALUEB'variables/36/.ATTRIBUTES/VARIABLE_VALUEB'variables/37/.ATTRIBUTES/VARIABLE_VALUEB'variables/38/.ATTRIBUTES/VARIABLE_VALUEB'variables/39/.ATTRIBUTES/VARIABLE_VALUEB'variables/40/.ATTRIBUTES/VARIABLE_VALUEB'variables/41/.ATTRIBUTES/VARIABLE_VALUEB'variables/42/.ATTRIBUTES/VARIABLE_VALUEB'variables/43/.ATTRIBUTES/VARIABLE_VALUEB'variables/44/.ATTRIBUTES/VARIABLE_VALUEB'variables/45/.ATTRIBUTES/VARIABLE_VALUEB'variables/46/.ATTRIBUTES/VARIABLE_VALUEB'variables/47/.ATTRIBUTES/VARIABLE_VALUEB'variables/48/.ATTRIBUTES/VARIABLE_VALUEB'variables/49/.ATTRIBUTES/VARIABLE_VALUEB'variables/50/.ATTRIBUTES/VARIABLE_VALUEB'variables/51/.ATTRIBUTES/VARIABLE_VALUEB'variables/52/.ATTRIBUTES/VARIABLE_VALUEB'variables/53/.ATTRIBUTES/VARIABLE_VALUEB'variables/54/.ATTRIBUTES/VARIABLE_VALUEB'variables/55/.ATTRIBUTES/VARIABLE_VALUEB'variables/56/.ATTRIBUTES/VARIABLE_VALUEB'variables/57/.ATTRIBUTES/VARIABLE_VALUEB'variables/58/.ATTRIBUTES/VARIABLE_VALUEB'variables/59/.ATTRIBUTES/VARIABLE_VALUEB'variables/60/.ATTRIBUTES/VARIABLE_VALUEB'variables/61/.ATTRIBUTES/VARIABLE_VALUEB'variables/62/.ATTRIBUTES/VARIABLE_VALUEB'variables/63/.ATTRIBUTES/VARIABLE_VALUEB'variables/64/.ATTRIBUTES/VARIABLE_VALUEB'variables/65/.ATTRIBUTES/VARIABLE_VALUEB'variables/66/.ATTRIBUTES/VARIABLE_VALUEB'variables/67/.ATTRIBUTES/VARIABLE_VALUEB'variables/68/.ATTRIBUTES/VARIABLE_VALUEB'variables/69/.ATTRIBUTES/VARIABLE_VALUEB'variables/70/.ATTRIBUTES/VARIABLE_VALUEB'variables/71/.ATTRIBUTES/VARIABLE_VALUEB'variables/72/.ATTRIBUTES/VARIABLE_VALUEB'variables/73/.ATTRIBUTES/VARIABLE_VALUEB'variables/74/.ATTRIBUTES/VARIABLE_VALUEB'variables/75/.ATTRIBUTES/VARIABLE_VALUEB'variables/76/.ATTRIBUTES/VARIABLE_VALUEB'variables/77/.ATTRIBUTES/VARIABLE_VALUEB'variables/78/.ATTRIBUTES/VARIABLE_VALUEB'variables/79/.ATTRIBUTES/VARIABLE_VALUEB'variables/80/.ATTRIBUTES/VARIABLE_VALUEB'variables/81/.ATTRIBUTES/VARIABLE_VALUEB'variables/82/.ATTRIBUTES/VARIABLE_VALUEB'variables/83/.ATTRIBUTES/VARIABLE_VALUEB'variables/84/.ATTRIBUTES/VARIABLE_VALUEB'variables/85/.ATTRIBUTES/VARIABLE_VALUEB'variables/86/.ATTRIBUTES/VARIABLE_VALUEB'variables/87/.ATTRIBUTES/VARIABLE_VALUEB'variables/88/.ATTRIBUTES/VARIABLE_VALUEB'variables/89/.ATTRIBUTES/VARIABLE_VALUEB'variables/90/.ATTRIBUTES/VARIABLE_VALUEB'variables/91/.ATTRIBUTES/VARIABLE_VALUEB'variables/92/.ATTRIBUTES/VARIABLE_VALUEB'variables/93/.ATTRIBUTES/VARIABLE_VALUEB'variables/94/.ATTRIBUTES/VARIABLE_VALUEB'variables/95/.ATTRIBUTES/VARIABLE_VALUEB'variables/96/.ATTRIBUTES/VARIABLE_VALUEB'variables/97/.ATTRIBUTES/VARIABLE_VALUEB'variables/98/.ATTRIBUTES/VARIABLE_VALUEB'variables/99/.ATTRIBUTES/VARIABLE_VALUEB(variables/100/.ATTRIBUTES/VARIABLE_VALUEB(variables/101/.ATTRIBUTES/VARIABLE_VALUEB(variables/102/.ATTRIBUTES/VARIABLE_VALUEB(variables/103/.ATTRIBUTES/VARIABLE_VALUEB(variables/104/.ATTRIBUTES/VARIABLE_VALUEB(variables/105/.ATTRIBUTES/VARIABLE_VALUEB(variables/106/.ATTRIBUTES/VARIABLE_VALUEB(variables/107/.ATTRIBUTES/VARIABLE_VALUEB(variables/108/.ATTRIBUTES/VARIABLE_VALUEB(variables/109/.ATTRIBUTES/VARIABLE_VALUEB(variables/110/.ATTRIBUTES/VARIABLE_VALUEB(variables/111/.ATTRIBUTES/VARIABLE_VALUEB(variables/112/.ATTRIBUTES/VARIABLE_VALUEB(variables/113/.ATTRIBUTES/VARIABLE_VALUEB(variables/114/.ATTRIBUTES/VARIABLE_VALUEB(variables/115/.ATTRIBUTES/VARIABLE_VALUEB(variables/116/.ATTRIBUTES/VARIABLE_VALUEB(variables/117/.ATTRIBUTES/VARIABLE_VALUEB(variables/118/.ATTRIBUTES/VARIABLE_VALUEB(variables/119/.ATTRIBUTES/VARIABLE_VALUEB(variables/120/.ATTRIBUTES/VARIABLE_VALUEB(variables/121/.ATTRIBUTES/VARIABLE_VALUEB(variables/122/.ATTRIBUTES/VARIABLE_VALUEB(variables/123/.ATTRIBUTES/VARIABLE_VALUEB(variables/124/.ATTRIBUTES/VARIABLE_VALUEB(variables/125/.ATTRIBUTES/VARIABLE_VALUEB(variables/126/.ATTRIBUTES/VARIABLE_VALUEB(variables/127/.ATTRIBUTES/VARIABLE_VALUEB(variables/128/.ATTRIBUTES/VARIABLE_VALUEB(variables/129/.ATTRIBUTES/VARIABLE_VALUEB(variables/130/.ATTRIBUTES/VARIABLE_VALUEB(variables/131/.ATTRIBUTES/VARIABLE_VALUEB(variables/132/.ATTRIBUTES/VARIABLE_VALUEB(variables/133/.ATTRIBUTES/VARIABLE_VALUEB(variables/134/.ATTRIBUTES/VARIABLE_VALUEB(variables/135/.ATTRIBUTES/VARIABLE_VALUEB(variables/136/.ATTRIBUTES/VARIABLE_VALUEB(variables/137/.ATTRIBUTES/VARIABLE_VALUEB(variables/138/.ATTRIBUTES/VARIABLE_VALUEB(variables/139/.ATTRIBUTES/VARIABLE_VALUEB(variables/140/.ATTRIBUTES/VARIABLE_VALUEB(variables/141/.ATTRIBUTES/VARIABLE_VALUEB(variables/142/.ATTRIBUTES/VARIABLE_VALUEB(variables/143/.ATTRIBUTES/VARIABLE_VALUEB(variables/144/.ATTRIBUTES/VARIABLE_VALUEB(variables/145/.ATTRIBUTES/VARIABLE_VALUEB(variables/146/.ATTRIBUTES/VARIABLE_VALUEB(variables/147/.ATTRIBUTES/VARIABLE_VALUEB(variables/148/.ATTRIBUTES/VARIABLE_VALUEB(variables/149/.ATTRIBUTES/VARIABLE_VALUEB(variables/150/.ATTRIBUTES/VARIABLE_VALUEB(variables/151/.ATTRIBUTES/VARIABLE_VALUEB(variables/152/.ATTRIBUTES/VARIABLE_VALUEB(variables/153/.ATTRIBUTES/VARIABLE_VALUEB(variables/154/.ATTRIBUTES/VARIABLE_VALUEB(variables/155/.ATTRIBUTES/VARIABLE_VALUEB(variables/156/.ATTRIBUTES/VARIABLE_VALUEB(variables/157/.ATTRIBUTES/VARIABLE_VALUEB(variables/158/.ATTRIBUTES/VARIABLE_VALUEB(variables/159/.ATTRIBUTES/VARIABLE_VALUEB(variables/160/.ATTRIBUTES/VARIABLE_VALUEB(variables/161/.ATTRIBUTES/VARIABLE_VALUEB(variables/162/.ATTRIBUTES/VARIABLE_VALUEB(variables/163/.ATTRIBUTES/VARIABLE_VALUEB(variables/164/.ATTRIBUTES/VARIABLE_VALUEB(variables/165/.ATTRIBUTES/VARIABLE_VALUEB(variables/166/.ATTRIBUTES/VARIABLE_VALUEB(variables/167/.ATTRIBUTES/VARIABLE_VALUEB(variables/168/.ATTRIBUTES/VARIABLE_VALUEB(variables/169/.ATTRIBUTES/VARIABLE_VALUEB(variables/170/.ATTRIBUTES/VARIABLE_VALUEB(variables/171/.ATTRIBUTES/VARIABLE_VALUEB(variables/172/.ATTRIBUTES/VARIABLE_VALUEB(variables/173/.ATTRIBUTES/VARIABLE_VALUEB(variables/174/.ATTRIBUTES/VARIABLE_VALUEB(variables/175/.ATTRIBUTES/VARIABLE_VALUEB(variables/176/.ATTRIBUTES/VARIABLE_VALUEB(variables/177/.ATTRIBUTES/VARIABLE_VALUEB(variables/178/.ATTRIBUTES/VARIABLE_VALUEB(variables/179/.ATTRIBUTES/VARIABLE_VALUEB(variables/180/.ATTRIBUTES/VARIABLE_VALUEB(variables/181/.ATTRIBUTES/VARIABLE_VALUEB(variables/182/.ATTRIBUTES/VARIABLE_VALUEB(variables/183/.ATTRIBUTES/VARIABLE_VALUEB(variables/184/.ATTRIBUTES/VARIABLE_VALUEB(variables/185/.ATTRIBUTES/VARIABLE_VALUEB(variables/186/.ATTRIBUTES/VARIABLE_VALUEB(variables/187/.ATTRIBUTES/VARIABLE_VALUEB(variables/188/.ATTRIBUTES/VARIABLE_VALUEB(variables/189/.ATTRIBUTES/VARIABLE_VALUEB(variables/190/.ATTRIBUTES/VARIABLE_VALUEB(variables/191/.ATTRIBUTES/VARIABLE_VALUEB(variables/192/.ATTRIBUTES/VARIABLE_VALUEB(variables/193/.ATTRIBUTES/VARIABLE_VALUEB(variables/194/.ATTRIBUTES/VARIABLE_VALUEB(variables/195/.ATTRIBUTES/VARIABLE_VALUEB(variables/196/.ATTRIBUTES/VARIABLE_VALUEB(variables/197/.ATTRIBUTES/VARIABLE_VALUEB(variables/198/.ATTRIBUTES/VARIABLE_VALUE2 SaveV2/tensor_names SaveV2/shape_and_slicesConst" /device:CPU:0* _output_shapes :Č* dtype0*¦ valueœB™ČBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB2 SaveV2/shape_and_slicesŪt SaveV2SaveV2ShardedFilename:filename:0SaveV2/tensor_names:output:0 SaveV2/shape_and_slices:output:0%savev2_variable_1_read_readvariableop5savev2_word_embeddings_embeddings_read_readvariableop8savev2_position_embedding_embeddings_read_readvariableop5savev2_type_embeddings_embeddings_read_readvariableop6savev2_embeddings_layer_norm_gamma_read_readvariableop5savev2_embeddings_layer_norm_beta_read_readvariableopJsavev2_transformer_layer_0_self_attention_query_kernel_read_readvariableopHsavev2_transformer_layer_0_self_attention_query_bias_read_readvariableopHsavev2_transformer_layer_0_self_attention_key_kernel_read_readvariableopFsavev2_transformer_layer_0_self_attention_key_bias_read_readvariableopJsavev2_transformer_layer_0_self_attention_value_kernel_read_readvariableopHsavev2_transformer_layer_0_self_attention_value_bias_read_readvariableopUsavev2_transformer_layer_0_self_attention_attention_output_kernel_read_readvariableopSsavev2_transformer_layer_0_self_attention_attention_output_bias_read_readvariableopNsavev2_transformer_layer_0_self_attention_layer_norm_gamma_read_readvariableopMsavev2_transformer_layer_0_self_attention_layer_norm_beta_read_readvariableopBsavev2_transformer_layer_0_intermediate_kernel_read_readvariableop@savev2_transformer_layer_0_intermediate_bias_read_readvariableop  _output_shapes :€:!?  _output_shapes :€:!@  _output_shapes :€:&A" _output_shapes : €€:!B  _output_shapes :€:&C" _output_shapes : €€:!D  _output_shapes :€:!E  _output_shapes :€:!F  _output_shapes :€:)G% # _output_shapes : € @:$H  _output_shapes : @:)I% # _output_shapes : € @:$J  _output_shapes : @:)K% # _output_shapes : € @:$L  _output_shapes : @:)M% # _output_shapes :  @€:!N  _output_shapes :€:!O  _output_shapes :€:!P  _output_shapes :€:&Q" _output_shapes : €€:!R  _output_shapes :€:&S" _output_shapes : €€:!T  _output_shapes :€:!U  _output_shapes :€:!V  _output_shapes :€:)W% # _output_shapes : € @:$X  _output_shapes : @:)Y% # _output_shapes : € @:$Z  _output_shapes : @:)[% # _output_shapes : € @:$\  _output_shapes : @:)]% # _output_shapes :  @€:!^  _output_shapes :€:!_  _output_shapes :€:!`  _output_shapes :€:&a" _output_shapes : €€:!b  _output_shapes :€:&c" _output_shapes : €€:!d  _output_shapes :€:!e  _output_shapes :€:!f  _output_shapes :€:)g% # _output_shapes : € @:$h  _output_shapes : @:)i% # _output_shapes : € @:$j  _output_shapes : @:)k% # _output_shapes : € @:$l  _output_shapes : @:)m% # _output_shapes :  @€:!n  _output_shapes :€:!o  _output_shapes :€:!p  _output_shapes :€:&q" _output_shapes : €€:!r  _output_shapes :€:&s" _output_shapes : €€:!t  _output_shapes :€:!u  _output_shapes :€:!v  _output_shapes :€:)w% # _output_shapes : € @:$x  _output_shapes : @:)y% # _output_shapes : € @:$z  _output_shapes : @:){% # _output_shapes : € @:$|  _output_shapes : @:)}% # _output_shapes :  @€:!~  _output_shapes :€:!  _output_shapes :€:"€  _output_shapes :€:'" _output_shapes : €€:"‚  _output_shapes :€:'ƒ" _output_shapes : €€:"„  _output_shapes :€:"…  _output_shapes :€:"†  _output_shapes :€:*‡% # _output_shapes : € @:%ˆ  _output_shapes : @:*‰% # _output_shapes : € @:%Š  _output_shapes : @:*‹% # _output_shapes : € @:%Œ  _output_shapes : @:*% # _output_shapes :  @€:"Ž  _output_shapes :€:"  _output_shapes :€:"  _output_shapes :€:'‘" _output_shapes : €€:"’  _output_shapes :€:'“" _output_shapes : €€:"”  _output_shapes :€:"•  _output_shapes :€:"–  _output_shapes :€:*—% # _output_shapes : € @:%˜  _output_shapes : @:*™% # _output_shapes : € @:%š  _output_shapes : @:*›% # _output_shapes : € @:%œ  _output_shapes : @:*% # _output_shapes :  @€:"ž  _output_shapes :€:"Ÿ  _output_shapes :€:"   _output_shapes :€:'”" _output_shapes : €€:"¢  _output_shapes :€:'£" _output_shapes : €€:"¤  _output_shapes :€:"„  _output_shapes :€:"¦  _output_shapes :€:*§% # _output_shapes : € @:%Ø  _output_shapes : @:*©% # _output_shapes : € @:%Ŗ  _output_shapes : @:*«% # _output_shapes : € @:%¬  _output_shapes : @:*­% # _output_shapes :  @€:"®  _output_shapes :€:"Æ  _output_shapes :€:"°  _output_shapes :€:'±" _output_shapes : €€:"²  _output_shapes :€:'³" _output_shapes : €€:"“  _output_shapes :€:"µ  _output_shapes :€:"¶  _output_shapes :€:*·% # _output_shapes : € @:%ø  _output_shapes : @:*¹% # _output_shapes : € @:%ŗ  _output_shapes : @:*»% # _output_shapes : € @:%¼  _output_shapes : @:*½% # _output_shapes :  @€:"¾  _output_shapes :€:"æ  _output_shapes :€:"Ą  _output_shapes :€:'Į" _output_shapes : €€:"Ā  _output_shapes :€:'Ć" _output_shapes : €€:"Ä  _output_shapes :€:"Å  _output_shapes :€:"Ę  _output_shapes :€:'Ē" _output_shapes : €€:"Č  _output_shapes :€:É  _output_shapes : § B &__inference_lambda_layer_call_fn_29263 inputs identity½ PartitionedCallPartitionedCallinputs* Tin 2* Tout 2* _collective_manager_ids *( _output_shapes : ’’’’’’’’’€* _read_only_resource_inputs ** config_proto  CPU  GPU2J8*J fERC A__inference_lambda_layer_call_and_return_conditional_losses_120072 PartitionedCallm IdentityIdentityPartitionedCall:output:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs š ` A__inference_dropout_layer_call_and_return_conditional_losses_7773 inputs identityˆc dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2 dropout/Const dropout/MulMulinputsdropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/MulT dropout/ShapeShapeinputs* T0* _output_shapes :2 dropout/ShapeĀ $dropout/random_uniform/RandomUniform RandomUniformdropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02& $dropout/random_uniform/RandomUniformu dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2 dropout/GreaterEqual/yĢ dropout/GreaterEqual GreaterEqual-dropout/random_uniform/RandomUniform:output:0dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/GreaterEqual dropout/CastCastdropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/Castˆ dropout/Mul_1Muldropout/Mul:z:0dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 dropout/Mul_1s IdentityIdentitydropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*4 _input_shapes# !: ’’’’’’’’’ ’’’’’’’’’€:]Y 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ _user_specified_nameinputs ö x >__inference_add_layer_call_and_return_conditional_losses_25395 inputs_0 inputs_1 inputs_2 identityg addAddV2inputs_0inputs_1* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 addj add_1AddV2add:z:0inputs_2* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 add_1k IdentityIdentity add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity" identityIdentity:output:0*v _input_shapese c: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€: ’’’’’’’’’ ’’’’’’’’’€:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/0:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/1:_[ 5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€ " _user_specified_name inputs/2 ©ļ ś˜ G__inference_functional_1_layer_call_and_return_conditional_losses_19407 inputs_0 inputs_1 inputs_27 3transformer_encoder_word_embeddings_gather_resourceB >transformer_encoder_position_embedding_readvariableop_resourceF Btransformer_encoder_type_embeddings_matmul_readvariableop_resourceS Otransformer_encoder_embeddings_layer_norm_batchnorm_mul_readvariableop_resourceO Ktransformer_encoder_embeddings_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_0_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_0_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_0_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_0_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_0_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_0_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_0_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_0_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_0_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_1_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_1_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_1_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_1_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_1_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_1_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_1_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_1_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_2_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_2_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_2_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_2_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_2_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_2_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_2_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_2_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_3_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_3_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_3_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_3_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_3_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_3_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_3_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_3_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_4_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_4_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_4_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_4_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_4_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_4_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_4_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_4_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_5_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_5_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_5_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_5_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_5_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_5_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_5_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_5_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_6_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_6_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_6_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_6_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_6_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_6_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_6_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_6_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_7_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_7_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_7_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_7_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_7_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_7_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_7_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_7_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_8_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_8_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_8_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_8_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_8_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_8_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_8_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_8_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_readvariableop_resourcef btransformer_encoder_transformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_9_self_attention_query_add_readvariableop_resourced `transformer_encoder_transformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resourceZ Vtransformer_encoder_transformer_layer_9_self_attention_key_add_readvariableop_resourcef btransformer_encoder_transformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource\ Xtransformer_encoder_transformer_layer_9_self_attention_value_add_readvariableop_resourceq mtransformer_encoder_transformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resourceg ctransformer_encoder_transformer_layer_9_self_attention_attention_output_add_readvariableop_resourcek gtransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceg ctransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resource^ Ztransformer_encoder_transformer_layer_9_intermediate_einsum_einsum_readvariableop_resourceT Ptransformer_encoder_transformer_layer_9_intermediate_add_readvariableop_resourceX Ttransformer_encoder_transformer_layer_9_output_einsum_einsum_readvariableop_resourceN Jtransformer_encoder_transformer_layer_9_output_add_readvariableop_resourcec _transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resource_ [transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_readvariableop_resourceg ctransformer_encoder_transformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_10_self_attention_query_add_readvariableop_resourcee atransformer_encoder_transformer_layer_10_self_attention_key_einsum_einsum_readvariableop_resource[ Wtransformer_encoder_transformer_layer_10_self_attention_key_add_readvariableop_resourceg ctransformer_encoder_transformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_10_self_attention_value_add_readvariableop_resourcer ntransformer_encoder_transformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resourceh dtransformer_encoder_transformer_layer_10_self_attention_attention_output_add_readvariableop_resourcel htransformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceh dtransformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resource_ [transformer_encoder_transformer_layer_10_intermediate_einsum_einsum_readvariableop_resourceU Qtransformer_encoder_transformer_layer_10_intermediate_add_readvariableop_resourceY Utransformer_encoder_transformer_layer_10_output_einsum_einsum_readvariableop_resourceO Ktransformer_encoder_transformer_layer_10_output_add_readvariableop_resourced `transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_mul_readvariableop_resource` \transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_readvariableop_resourceg ctransformer_encoder_transformer_layer_11_self_attention_query_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_11_self_attention_query_add_readvariableop_resourcee atransformer_encoder_transformer_layer_11_self_attention_key_einsum_einsum_readvariableop_resource[ Wtransformer_encoder_transformer_layer_11_self_attention_key_add_readvariableop_resourceg ctransformer_encoder_transformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource] Ytransformer_encoder_transformer_layer_11_self_attention_value_add_readvariableop_resourcer ntransformer_encoder_transformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resourceh dtransformer_encoder_transformer_layer_11_self_attention_attention_output_add_readvariableop_resourcel htransformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resourceh dtransformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resource_ [transformer_encoder_transformer_layer_11_intermediate_einsum_einsum_readvariableop_resourceU Qtransformer_encoder_transformer_layer_11_intermediate_add_readvariableop_resourceY Utransformer_encoder_transformer_layer_11_output_einsum_einsum_readvariableop_resourceO Ktransformer_encoder_transformer_layer_11_output_add_readvariableop_resourced `transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_mul_readvariableop_resource` \transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_readvariableop_resourceG Ctransformer_encoder_pooler_transform_matmul_readvariableop_resourceH Dtransformer_encoder_pooler_transform_biasadd_readvariableop_resource identity identity_1ˆ¹ 1transformer_encoder/word_embeddings/Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’23 1transformer_encoder/word_embeddings/Reshape/shapeŁ +transformer_encoder/word_embeddings/ReshapeReshapeinputs_0:transformer_encoder/word_embeddings/Reshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2- +transformer_encoder/word_embeddings/Reshapeœ *transformer_encoder/word_embeddings/GatherResourceGather3transformer_encoder_word_embeddings_gather_resource4transformer_encoder/word_embeddings/Reshape:output:0* Tindices0*( _output_shapes : ’’’’’’’’’€* dtype02, *transformer_encoder/word_embeddings/GatherŠ ,transformer_encoder/word_embeddings/IdentityIdentity3transformer_encoder/word_embeddings/Gather:output:0* T0*( _output_shapes : ’’’’’’’’’€2. ,transformer_encoder/word_embeddings/IdentityŽ )transformer_encoder/word_embeddings/ShapeShapeinputs_0* T0* _output_shapes :2+ )transformer_encoder/word_embeddings/Shapeµ 3transformer_encoder/word_embeddings/concat/values_1Const* _output_shapes :* dtype0* valueB :€25 3transformer_encoder/word_embeddings/concat/values_1¤ /transformer_encoder/word_embeddings/concat/axisConst* _output_shapes :* dtype0* value B:21 /transformer_encoder/word_embeddings/concat/axis¾ *transformer_encoder/word_embeddings/concatConcatV22transformer_encoder/word_embeddings/Shape:output:0 transformer_encoder/position_embedding/strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2@ >transformer_encoder/position_embedding/strided_slice_1/stack_1Ź >transformer_encoder/position_embedding/strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2@ >transformer_encoder/position_embedding/strided_slice_1/stack_2Ö 6transformer_encoder/position_embedding/strided_slice_1 StridedSlice5transformer_encoder/position_embedding/Shape:output:0Etransformer_encoder/position_embedding/strided_slice_1/stack:output:0Gtransformer_encoder/position_embedding/strided_slice_1/stack_1:output:0Gtransformer_encoder/position_embedding/strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask28 6transformer_encoder/position_embedding/strided_slice_1ļ 5transformer_encoder/position_embedding/ReadVariableOpReadVariableOp>transformer_encoder_position_embedding_readvariableop_resource* _output_shapes : €€* dtype027 5transformer_encoder/position_embedding/ReadVariableOpĶ transformer_encoder/position_embedding/strided_slice_2/stack_1Pack?transformer_encoder/position_embedding/strided_slice_1:output:0Itransformer_encoder/position_embedding/strided_slice_2/stack_1/1:output:0* N* T0* _output_shapes :2@ >transformer_encoder/position_embedding/strided_slice_2/stack_1Ń >transformer_encoder/position_embedding/strided_slice_2/stack_2Const* _output_shapes :* dtype0* valueB"2@ >transformer_encoder/position_embedding/strided_slice_2/stack_2ś 6transformer_encoder/position_embedding/strided_slice_2 StridedSlice=transformer_encoder/position_embedding/ReadVariableOp:value:0Etransformer_encoder/position_embedding/strided_slice_2/stack:output:0Gtransformer_encoder/position_embedding/strided_slice_2/stack_1:output:0Gtransformer_encoder/position_embedding/strided_slice_2/stack_2:output:0* Index0* T0*( _output_shapes : ’’’’’’’’’€* begin_mask* end_mask28 6transformer_encoder/position_embedding/strided_slice_2» :transformer_encoder/position_embedding/BroadcastTo/shape/2Const* _output_shapes :* dtype0* value B:€2< :transformer_encoder/position_embedding/BroadcastTo/shape/2ļ 8transformer_encoder/position_embedding/BroadcastTo/shapePack=transformer_encoder/position_embedding/strided_slice:output:0?transformer_encoder/position_embedding/strided_slice_1:output:0Ctransformer_encoder/position_embedding/BroadcastTo/shape/2:output:0* N* T0* _output_shapes :2: 8transformer_encoder/position_embedding/BroadcastTo/shape» 2transformer_encoder/position_embedding/BroadcastTo BroadcastTo?transformer_encoder/position_embedding/strided_slice_2:output:0Atransformer_encoder/position_embedding/BroadcastTo/shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€24 2transformer_encoder/position_embedding/BroadcastTo¹ 1transformer_encoder/type_embeddings/Reshape/shapeConst* _output_shapes :* dtype0* valueB: ’’’’’’’’’23 1transformer_encoder/type_embeddings/Reshape/shapeŁ +transformer_encoder/type_embeddings/ReshapeReshapeinputs_2:transformer_encoder/type_embeddings/Reshape/shape:output:0* T0*# _output_shapes :  ’’’’’’’’’2- +transformer_encoder/type_embeddings/Reshape± 4transformer_encoder/type_embeddings/one_hot/on_valueConst* _output_shapes :* dtype0* value B *€?26 4transformer_encoder/type_embeddings/one_hot/on_value³ 5transformer_encoder/type_embeddings/one_hot/off_valueConst* _output_shapes :* dtype0* value B *27 5transformer_encoder/type_embeddings/one_hot/off_valueØ 1transformer_encoder/type_embeddings/one_hot/depthConst* _output_shapes :* dtype0* value B:23 1transformer_encoder/type_embeddings/one_hot/depth‘ +transformer_encoder/type_embeddings/one_hotOneHot4transformer_encoder/type_embeddings/Reshape:output:0:transformer_encoder/type_embeddings/one_hot/depth:output:0=transformer_encoder/type_embeddings/one_hot/on_value:output:0>transformer_encoder/type_embeddings/one_hot/off_value:output:0* T0* TI0*' _output_shapes : ’’’’’’’’’2- +transformer_encoder/type_embeddings/one_hotś 9transformer_encoder/type_embeddings/MatMul/ReadVariableOpReadVariableOpBtransformer_encoder_type_embeddings_matmul_readvariableop_resource* _output_shapes : €* dtype02; 9transformer_encoder/type_embeddings/MatMul/ReadVariableOpŽ *transformer_encoder/type_embeddings/MatMulMatMul4transformer_encoder/type_embeddings/one_hot:output:0Atransformer_encoder/type_embeddings/MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2, *transformer_encoder/type_embeddings/MatMulŽ )transformer_encoder/type_embeddings/ShapeShapeinputs_2* T0* _output_shapes :2+ )transformer_encoder/type_embeddings/Shapeµ 3transformer_encoder/type_embeddings/concat/values_1Const* _output_shapes :* dtype0* valueB :€25 3transformer_encoder/type_embeddings/concat/values_1¤ /transformer_encoder/type_embeddings/concat/axisConst* _output_shapes :* dtype0* value B:21 /transformer_encoder/type_embeddings/concat/axis¾ *transformer_encoder/type_embeddings/concatConcatV22transformer_encoder/type_embeddings/Shape:output:0transformer_encoder/embeddings/layer_norm/moments/StopGradient StopGradient?transformer_encoder/embeddings/layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2@ >transformer_encoder/embeddings/layer_norm/moments/StopGradientĖ Ctransformer_encoder/embeddings/layer_norm/moments/SquaredDifferenceSquaredDifference!transformer_encoder/add/add_1:z:0Gtransformer_encoder/embeddings/layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2E Ctransformer_encoder/embeddings/layer_norm/moments/SquaredDifferenceę Ltransformer_encoder/embeddings/layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2N Ltransformer_encoder/embeddings/layer_norm/moments/variance/reduction_indicesš :transformer_encoder/embeddings/layer_norm/moments/varianceMeanGtransformer_encoder/embeddings/layer_norm/moments/SquaredDifference:z:0Utransformer_encoder/embeddings/layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2< :transformer_encoder/embeddings/layer_norm/moments/variance» 9transformer_encoder/embeddings/layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2; 9transformer_encoder/embeddings/layer_norm/batchnorm/add/yĆ 7transformer_encoder/embeddings/layer_norm/batchnorm/addAddV2Ctransformer_encoder/embeddings/layer_norm/moments/variance:output:0Btransformer_encoder/embeddings/layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’29 7transformer_encoder/embeddings/layer_norm/batchnorm/addū 9transformer_encoder/embeddings/layer_norm/batchnorm/RsqrtRsqrt;transformer_encoder/embeddings/layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2; 9transformer_encoder/embeddings/layer_norm/batchnorm/Rsqrt Ftransformer_encoder/embeddings/layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpOtransformer_encoder_embeddings_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02H Ftransformer_encoder/embeddings/layer_norm/batchnorm/mul/ReadVariableOpČ 7transformer_encoder/embeddings/layer_norm/batchnorm/mulMul=transformer_encoder/embeddings/layer_norm/batchnorm/Rsqrt:y:0Ntransformer_encoder/embeddings/layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer_encoder/embeddings/layer_norm/batchnorm/mul 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_1Mul!transformer_encoder/add/add_1:z:0;transformer_encoder/embeddings/layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_1» 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_2Mul?transformer_encoder/embeddings/layer_norm/moments/mean:output:0;transformer_encoder/embeddings/layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/embeddings/layer_norm/batchnorm/mul_2‘ Btransformer_encoder/embeddings/layer_norm/batchnorm/ReadVariableOpReadVariableOpKtransformer_encoder_embeddings_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02D Btransformer_encoder/embeddings/layer_norm/batchnorm/ReadVariableOpÄ 7transformer_encoder/embeddings/layer_norm/batchnorm/subSubJtransformer_encoder/embeddings/layer_norm/batchnorm/ReadVariableOp:value:0=transformer_encoder/embeddings/layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€29 7transformer_encoder/embeddings/layer_norm/batchnorm/sub» 9transformer_encoder/embeddings/layer_norm/batchnorm/add_1AddV2=transformer_encoder/embeddings/layer_norm/batchnorm/mul_1:z:0;transformer_encoder/embeddings/layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/embeddings/layer_norm/batchnorm/add_1› )transformer_encoder/dropout/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2+ )transformer_encoder/dropout/dropout/ConstŒ 'transformer_encoder/dropout/dropout/MulMul=transformer_encoder/embeddings/layer_norm/batchnorm/add_1:z:02transformer_encoder/dropout/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2) 'transformer_encoder/dropout/dropout/MulĆ )transformer_encoder/dropout/dropout/ShapeShape=transformer_encoder/embeddings/layer_norm/batchnorm/add_1:z:0* T0* _output_shapes :2+ )transformer_encoder/dropout/dropout/Shape– @transformer_encoder/dropout/dropout/random_uniform/RandomUniform RandomUniform2transformer_encoder/dropout/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02B @transformer_encoder/dropout/dropout/random_uniform/RandomUniform­ 2transformer_encoder/dropout/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=24 2transformer_encoder/dropout/dropout/GreaterEqual/y¼ 0transformer_encoder/dropout/dropout/GreaterEqual GreaterEqualItransformer_encoder/dropout/dropout/random_uniform/RandomUniform:output:0;transformer_encoder/dropout/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€22 0transformer_encoder/dropout/dropout/GreaterEqualį (transformer_encoder/dropout/dropout/CastCast4transformer_encoder/dropout/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2* (transformer_encoder/dropout/dropout/Castų )transformer_encoder/dropout/dropout/Mul_1Mul+transformer_encoder/dropout/dropout/Mul:z:0,transformer_encoder/dropout/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2+ )transformer_encoder/dropout/dropout/Mul_1» -transformer_encoder/self_attention_mask/ShapeShape-transformer_encoder/dropout/dropout/Mul_1:z:0* T0* _output_shapes :2/ -transformer_encoder/self_attention_mask/ShapeÄ ;transformer_encoder/self_attention_mask/strided_slice/stackConst* _output_shapes :* dtype0* value B :2= ;transformer_encoder/self_attention_mask/strided_slice/stackČ =transformer_encoder/self_attention_mask/strided_slice/stack_1Const* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice/stack_1Č =transformer_encoder/self_attention_mask/strided_slice/stack_2Const* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice/stack_2Ņ 5transformer_encoder/self_attention_mask/strided_slice StridedSlice6transformer_encoder/self_attention_mask/Shape:output:0Dtransformer_encoder/self_attention_mask/strided_slice/stack:output:0Ftransformer_encoder/self_attention_mask/strided_slice/stack_1:output:0Ftransformer_encoder/self_attention_mask/strided_slice/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask27 5transformer_encoder/self_attention_mask/strided_sliceČ =transformer_encoder/self_attention_mask/strided_slice_1/stackConst* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice_1/stackĢ ?transformer_encoder/self_attention_mask/strided_slice_1/stack_1Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_1/stack_1Ģ ?transformer_encoder/self_attention_mask/strided_slice_1/stack_2Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_1/stack_2Ü 7transformer_encoder/self_attention_mask/strided_slice_1 StridedSlice6transformer_encoder/self_attention_mask/Shape:output:0Ftransformer_encoder/self_attention_mask/strided_slice_1/stack:output:0Htransformer_encoder/self_attention_mask/strided_slice_1/stack_1:output:0Htransformer_encoder/self_attention_mask/strided_slice_1/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask29 7transformer_encoder/self_attention_mask/strided_slice_1š /transformer_encoder/self_attention_mask/Shape_1Shapeinputs_1* T0* _output_shapes :21 /transformer_encoder/self_attention_mask/Shape_1Č =transformer_encoder/self_attention_mask/strided_slice_2/stackConst* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice_2/stackĢ ?transformer_encoder/self_attention_mask/strided_slice_2/stack_1Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_2/stack_1Ģ ?transformer_encoder/self_attention_mask/strided_slice_2/stack_2Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_2/stack_2Ž 7transformer_encoder/self_attention_mask/strided_slice_2 StridedSlice8transformer_encoder/self_attention_mask/Shape_1:output:0Ftransformer_encoder/self_attention_mask/strided_slice_2/stack:output:0Htransformer_encoder/self_attention_mask/strided_slice_2/stack_1:output:0Htransformer_encoder/self_attention_mask/strided_slice_2/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask29 7transformer_encoder/self_attention_mask/strided_slice_2Č =transformer_encoder/self_attention_mask/strided_slice_3/stackConst* _output_shapes :* dtype0* value B :2? =transformer_encoder/self_attention_mask/strided_slice_3/stackĢ ?transformer_encoder/self_attention_mask/strided_slice_3/stack_1Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_3/stack_1Ģ ?transformer_encoder/self_attention_mask/strided_slice_3/stack_2Const* _output_shapes :* dtype0* value B :2A ?transformer_encoder/self_attention_mask/strided_slice_3/stack_2Ž 7transformer_encoder/self_attention_mask/strided_slice_3 StridedSlice8transformer_encoder/self_attention_mask/Shape_1:output:0Ftransformer_encoder/self_attention_mask/strided_slice_3/stack:output:0Htransformer_encoder/self_attention_mask/strided_slice_3/stack_1:output:0Htransformer_encoder/self_attention_mask/strided_slice_3/stack_2:output:0* Index0* T0* _output_shapes :* shrink_axis_mask29 7transformer_encoder/self_attention_mask/strided_slice_3“ 7transformer_encoder/self_attention_mask/Reshape/shape/1Const* _output_shapes :* dtype0* value B:29 7transformer_encoder/self_attention_mask/Reshape/shape/1č 5transformer_encoder/self_attention_mask/Reshape/shapePack>transformer_encoder/self_attention_mask/strided_slice:output:0@transformer_encoder/self_attention_mask/Reshape/shape/1:output:0@transformer_encoder/self_attention_mask/strided_slice_3:output:0* N* T0* _output_shapes :27 5transformer_encoder/self_attention_mask/Reshape/shapeö /transformer_encoder/self_attention_mask/ReshapeReshapeinputs_1>transformer_encoder/self_attention_mask/Reshape/shape:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’21 /transformer_encoder/self_attention_mask/Reshapeģ ,transformer_encoder/self_attention_mask/CastCast8transformer_encoder/self_attention_mask/Reshape:output:0* DstT0* SrcT0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2. ,transformer_encoder/self_attention_mask/CastŽ 0transformer_encoder/self_attention_mask/ones/mulMul>transformer_encoder/self_attention_mask/strided_slice:output:0@transformer_encoder/self_attention_mask/strided_slice_1:output:0* T0* _output_shapes :22 0transformer_encoder/self_attention_mask/ones/mul® 4transformer_encoder/self_attention_mask/ones/mul_1/yConst* _output_shapes :* dtype0* value B:26 4transformer_encoder/self_attention_mask/ones/mul_1/y… 2transformer_encoder/self_attention_mask/ones/mul_1Mul4transformer_encoder/self_attention_mask/ones/mul:z:0=transformer_encoder/self_attention_mask/ones/mul_1/y:output:0* T0* _output_shapes :24 2transformer_encoder/self_attention_mask/ones/mul_1­ 3transformer_encoder/self_attention_mask/ones/Less/yConst* _output_shapes :* dtype0* value B:č25 3transformer_encoder/self_attention_mask/ones/Less/y… 1transformer_encoder/self_attention_mask/ones/LessLess6transformer_encoder/self_attention_mask/ones/mul_1:z:0transformer_encoder/self_attention_mask/strided_slice:output:0@transformer_encoder/self_attention_mask/strided_slice_1:output:0>transformer_encoder/self_attention_mask/ones/packed/2:output:0* N* T0* _output_shapes :25 3transformer_encoder/self_attention_mask/ones/packed­ 2transformer_encoder/self_attention_mask/ones/ConstConst* _output_shapes :* dtype0* value B *€?24 2transformer_encoder/self_attention_mask/ones/Constž ,transformer_encoder/self_attention_mask/onesFillabde2L Jtransformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_0_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_0/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_0/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_0/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_0/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_0_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp— Htransformer_encoder/transformer/layer_0/self_attention/key/einsum/EinsumEinsum-transformer_encoder/dropout/dropout/Mul_1:z:0_transformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_0_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_0/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_0/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_0/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_0/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_0_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp Jtransformer_encoder/transformer/layer_0/self_attention/value/einsum/EinsumEinsum-transformer_encoder/dropout/dropout/Mul_1:z:0atransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_0_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_0/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_0/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_0/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_0/self_attention/value/add” Dtransformer_encoder/transformer/layer_0/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_0/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_0/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_0/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_0/self_attention/Mul:z:0Mtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_0/self_attention/masked_softmax/addÄ Mtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/SoftmaxSoftmaxMtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/Softmaxį Ltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2N Ltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Const› Jtransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/MulMulWtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/Softmax:softmax:0Utransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Mul£ Ltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/ShapeShapeWtransformer_encoder/transformer/layer_0/self_attention/masked_softmax/Softmax:softmax:0* T0* _output_shapes :2N Ltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Shape‹ ctransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/random_uniform/RandomUniform RandomUniformUtransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02e ctransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/random_uniform/RandomUniformó Utransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2W Utransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/GreaterEqual/yŌ Stransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/GreaterEqual GreaterEqualltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/random_uniform/RandomUniform:output:0^transformer_encoder/transformer/layer_0/self_attention/dropout/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2U Stransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/GreaterEqualÖ Ktransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/CastCastWtransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Cast Ltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Mul_1MulNtransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Mul:z:0Otransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2N Ltransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Mul_1 Ftransformer_encoder/transformer/layer_0/self_attention/einsum_1/EinsumEinsumPtransformer_encoder/transformer/layer_0/self_attention/dropout/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_0/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_0/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_0_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_0/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_0_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_0/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_0/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_0/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_0/self_attention/attention_output/addĒ ?transformer_encoder/transformer/layer_0/dropout_1/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2A ?transformer_encoder/transformer/layer_0/dropout_1/dropout/Constą =transformer_encoder/transformer/layer_0/dropout_1/dropout/MulMulOtransformer_encoder/transformer/layer_0/self_attention/attention_output/add:z:0Htransformer_encoder/transformer/layer_0/dropout_1/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer_encoder/transformer/layer_0/dropout_1/dropout/Mul ?transformer_encoder/transformer/layer_0/dropout_1/dropout/ShapeShapeOtransformer_encoder/transformer/layer_0/self_attention/attention_output/add:z:0* T0* _output_shapes :2A ?transformer_encoder/transformer/layer_0/dropout_1/dropout/ShapeŲ Vtransformer_encoder/transformer/layer_0/dropout_1/dropout/random_uniform/RandomUniform RandomUniformHtransformer_encoder/transformer/layer_0/dropout_1/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02X Vtransformer_encoder/transformer/layer_0/dropout_1/dropout/random_uniform/RandomUniformŁ Htransformer_encoder/transformer/layer_0/dropout_1/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2J Htransformer_encoder/transformer/layer_0/dropout_1/dropout/GreaterEqual/y” Ftransformer_encoder/transformer/layer_0/dropout_1/dropout/GreaterEqual GreaterEqual_transformer_encoder/transformer/layer_0/dropout_1/dropout/random_uniform/RandomUniform:output:0Qtransformer_encoder/transformer/layer_0/dropout_1/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ftransformer_encoder/transformer/layer_0/dropout_1/dropout/GreaterEqual£ >transformer_encoder/transformer/layer_0/dropout_1/dropout/CastCastJtransformer_encoder/transformer/layer_0/dropout_1/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_0/dropout_1/dropout/CastŠ ?transformer_encoder/transformer/layer_0/dropout_1/dropout/Mul_1MulAtransformer_encoder/transformer/layer_0/dropout_1/dropout/Mul:z:0Btransformer_encoder/transformer/layer_0/dropout_1/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_0/dropout_1/dropout/Mul_1— +transformer_encoder/transformer/layer_0/addAddV2-transformer_encoder/dropout/dropout/Mul_1:z:0Ctransformer_encoder/transformer/layer_0/dropout_1/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_0/addŽ `transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_0/add:z:0itransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_0/add:z:0_transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_0/add:z:0Stransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_0/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_0_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_0_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_0/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_0_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_0/intermediate/addAddV2Ktransformer_encoder/transformer/layer_0/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_0/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_0/intermediate/add¹ 8transformer_encoder/transformer/layer_0/activation/Pow/yConst* _output_shapes :* dtype0* value B *@@2: 8transformer_encoder/transformer/layer_0/activation/Pow/yø 6transformer_encoder/transformer/layer_0/activation/PowPowabd2> transformer_encoder/transformer/layer_0/dropout_2/dropout/CastCastJtransformer_encoder/transformer/layer_0/dropout_2/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_0/dropout_2/dropout/CastŠ ?transformer_encoder/transformer/layer_0/dropout_2/dropout/Mul_1MulAtransformer_encoder/transformer/layer_0/dropout_2/dropout/Mul:z:0Btransformer_encoder/transformer/layer_0/dropout_2/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_0/dropout_2/dropout/Mul_1Ć -transformer_encoder/transformer/layer_0/add_1AddV2Ctransformer_encoder/transformer/layer_0/dropout_2/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_0/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_0/add_1ž Xtransformer_encoder/transformer/layer_0/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_0/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_0/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_0/add_1:z:0atransformer_encoder/transformer/layer_0/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_0/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_0/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_0/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_0/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_0/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_0/add_1:z:0Wtransformer_encoder/transformer/layer_0/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_0/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_0/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_0/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_0/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_0/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_0/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_0/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_0/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_0/add_1:z:0Ktransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_0/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_0_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_1_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_1/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_1_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_1/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_1/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_1/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_1/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_1_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_1/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_1_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_1/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_1/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_1/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_1/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_1_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_1/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_1_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_1/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_1/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_1/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_1/self_attention/value/add” Dtransformer_encoder/transformer/layer_1/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_1/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_1/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_1/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_1/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/addŹ Otransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/Softmaxå Ntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2P Ntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Const£ Ltransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/MulMulYtransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/Softmax:softmax:0Wtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2N Ltransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Mul© Ntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/ShapeShapeYtransformer_encoder/transformer/layer_1/self_attention/masked_softmax_1/Softmax:softmax:0* T0* _output_shapes :2P Ntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Shape‘ etransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/random_uniform/RandomUniform RandomUniformWtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02g etransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/random_uniform/RandomUniform÷ Wtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Y Wtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual/yÜ Utransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual GreaterEqualntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/random_uniform/RandomUniform:output:0`transformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2W Utransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/GreaterEqualÜ Mtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/CastCastYtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Cast˜ Ntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Mul_1MulPtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Mul:z:0Qtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Mul_1Ÿ Ftransformer_encoder/transformer/layer_1/self_attention/einsum_1/EinsumEinsumRtransformer_encoder/transformer/layer_1/self_attention/dropout_3/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_1/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_1/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_1_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_1/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_1_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_1/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_1/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_1/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_1/self_attention/attention_output/addĒ ?transformer_encoder/transformer/layer_1/dropout_4/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2A ?transformer_encoder/transformer/layer_1/dropout_4/dropout/Constą =transformer_encoder/transformer/layer_1/dropout_4/dropout/MulMulOtransformer_encoder/transformer/layer_1/self_attention/attention_output/add:z:0Htransformer_encoder/transformer/layer_1/dropout_4/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer_encoder/transformer/layer_1/dropout_4/dropout/Mul ?transformer_encoder/transformer/layer_1/dropout_4/dropout/ShapeShapeOtransformer_encoder/transformer/layer_1/self_attention/attention_output/add:z:0* T0* _output_shapes :2A ?transformer_encoder/transformer/layer_1/dropout_4/dropout/ShapeŲ Vtransformer_encoder/transformer/layer_1/dropout_4/dropout/random_uniform/RandomUniform RandomUniformHtransformer_encoder/transformer/layer_1/dropout_4/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02X Vtransformer_encoder/transformer/layer_1/dropout_4/dropout/random_uniform/RandomUniformŁ Htransformer_encoder/transformer/layer_1/dropout_4/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2J Htransformer_encoder/transformer/layer_1/dropout_4/dropout/GreaterEqual/y” Ftransformer_encoder/transformer/layer_1/dropout_4/dropout/GreaterEqual GreaterEqual_transformer_encoder/transformer/layer_1/dropout_4/dropout/random_uniform/RandomUniform:output:0Qtransformer_encoder/transformer/layer_1/dropout_4/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ftransformer_encoder/transformer/layer_1/dropout_4/dropout/GreaterEqual£ >transformer_encoder/transformer/layer_1/dropout_4/dropout/CastCastJtransformer_encoder/transformer/layer_1/dropout_4/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_1/dropout_4/dropout/CastŠ ?transformer_encoder/transformer/layer_1/dropout_4/dropout/Mul_1MulAtransformer_encoder/transformer/layer_1/dropout_4/dropout/Mul:z:0Btransformer_encoder/transformer/layer_1/dropout_4/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_1/dropout_4/dropout/Mul_1· +transformer_encoder/transformer/layer_1/addAddV2Mtransformer_encoder/transformer/layer_0/output_layer_norm/batchnorm/add_1:z:0Ctransformer_encoder/transformer/layer_1/dropout_4/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_1/addŽ `transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_1/add:z:0itransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_1/add:z:0_transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_1/add:z:0Stransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_1/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_1_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_1_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_1/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_1_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_1/intermediate/addAddV2Ktransformer_encoder/transformer/layer_1/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_1/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_1/intermediate/add½ :transformer_encoder/transformer/layer_1/activation_1/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_1/activation_1/Pow/y¾ 8transformer_encoder/transformer/layer_1/activation_1/PowPow transformer_encoder/transformer/layer_1/activation_1/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_1/activation_1/TanhĮ transformer_encoder/transformer/layer_1/activation_1/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_1/activation_1/mul_2½ :transformer_encoder/transformer/layer_1/activation_1/mul_3Multransformer_encoder/transformer/layer_1/activation_1/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_1/activation_1/mul_3± Ktransformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_1_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_1/activation_1/mul_3:z:0Stransformer_encoder/transformer/layer_1/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_1/dropout_5/dropout/CastCastJtransformer_encoder/transformer/layer_1/dropout_5/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_1/dropout_5/dropout/CastŠ ?transformer_encoder/transformer/layer_1/dropout_5/dropout/Mul_1MulAtransformer_encoder/transformer/layer_1/dropout_5/dropout/Mul:z:0Btransformer_encoder/transformer/layer_1/dropout_5/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_1/dropout_5/dropout/Mul_1Ć -transformer_encoder/transformer/layer_1/add_1AddV2Ctransformer_encoder/transformer/layer_1/dropout_5/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_1/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_1/add_1ž Xtransformer_encoder/transformer/layer_1/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_1/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_1/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_1/add_1:z:0atransformer_encoder/transformer/layer_1/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_1/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_1/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_1/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_1/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_1/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_1/add_1:z:0Wtransformer_encoder/transformer/layer_1/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_1/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_1/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_1/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_1/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_1/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_1/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_1/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_1/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_1/add_1:z:0Ktransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_1/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_1_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_2_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_2/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_2_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_2/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_2/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_2/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_2/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_2_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_2/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_2_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_2/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_2/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_2/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_2/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_2_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_2/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_2_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_2/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_2/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_2/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_2/self_attention/value/add” Dtransformer_encoder/transformer/layer_2/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_2/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_2/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_2/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_2/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/addŹ Otransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/Softmaxå Ntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2P Ntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Const£ Ltransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/MulMulYtransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/Softmax:softmax:0Wtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2N Ltransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Mul© Ntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/ShapeShapeYtransformer_encoder/transformer/layer_2/self_attention/masked_softmax_2/Softmax:softmax:0* T0* _output_shapes :2P Ntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Shape‘ etransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/random_uniform/RandomUniform RandomUniformWtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02g etransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/random_uniform/RandomUniform÷ Wtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Y Wtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual/yÜ Utransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual GreaterEqualntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/random_uniform/RandomUniform:output:0`transformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2W Utransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/GreaterEqualÜ Mtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/CastCastYtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Cast˜ Ntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Mul_1MulPtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Mul:z:0Qtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Mul_1Ÿ Ftransformer_encoder/transformer/layer_2/self_attention/einsum_1/EinsumEinsumRtransformer_encoder/transformer/layer_2/self_attention/dropout_6/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_2/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_2/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_2_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_2/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_2_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_2/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_2/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_2/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_2/self_attention/attention_output/addĒ ?transformer_encoder/transformer/layer_2/dropout_7/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2A ?transformer_encoder/transformer/layer_2/dropout_7/dropout/Constą =transformer_encoder/transformer/layer_2/dropout_7/dropout/MulMulOtransformer_encoder/transformer/layer_2/self_attention/attention_output/add:z:0Htransformer_encoder/transformer/layer_2/dropout_7/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2? =transformer_encoder/transformer/layer_2/dropout_7/dropout/Mul ?transformer_encoder/transformer/layer_2/dropout_7/dropout/ShapeShapeOtransformer_encoder/transformer/layer_2/self_attention/attention_output/add:z:0* T0* _output_shapes :2A ?transformer_encoder/transformer/layer_2/dropout_7/dropout/ShapeŲ Vtransformer_encoder/transformer/layer_2/dropout_7/dropout/random_uniform/RandomUniform RandomUniformHtransformer_encoder/transformer/layer_2/dropout_7/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02X Vtransformer_encoder/transformer/layer_2/dropout_7/dropout/random_uniform/RandomUniformŁ Htransformer_encoder/transformer/layer_2/dropout_7/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2J Htransformer_encoder/transformer/layer_2/dropout_7/dropout/GreaterEqual/y” Ftransformer_encoder/transformer/layer_2/dropout_7/dropout/GreaterEqual GreaterEqual_transformer_encoder/transformer/layer_2/dropout_7/dropout/random_uniform/RandomUniform:output:0Qtransformer_encoder/transformer/layer_2/dropout_7/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2H Ftransformer_encoder/transformer/layer_2/dropout_7/dropout/GreaterEqual£ >transformer_encoder/transformer/layer_2/dropout_7/dropout/CastCastJtransformer_encoder/transformer/layer_2/dropout_7/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_2/dropout_7/dropout/CastŠ ?transformer_encoder/transformer/layer_2/dropout_7/dropout/Mul_1MulAtransformer_encoder/transformer/layer_2/dropout_7/dropout/Mul:z:0Btransformer_encoder/transformer/layer_2/dropout_7/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_2/dropout_7/dropout/Mul_1· +transformer_encoder/transformer/layer_2/addAddV2Mtransformer_encoder/transformer/layer_1/output_layer_norm/batchnorm/add_1:z:0Ctransformer_encoder/transformer/layer_2/dropout_7/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_2/addŽ `transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_2/add:z:0itransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_2/add:z:0_transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_2/add:z:0Stransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_2/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_2_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_2_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_2/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_2_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_2/intermediate/addAddV2Ktransformer_encoder/transformer/layer_2/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_2/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_2/intermediate/add½ :transformer_encoder/transformer/layer_2/activation_2/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_2/activation_2/Pow/y¾ 8transformer_encoder/transformer/layer_2/activation_2/PowPow transformer_encoder/transformer/layer_2/activation_2/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_2/activation_2/TanhĮ transformer_encoder/transformer/layer_2/activation_2/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_2/activation_2/mul_2½ :transformer_encoder/transformer/layer_2/activation_2/mul_3Multransformer_encoder/transformer/layer_2/activation_2/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_2/activation_2/mul_3± Ktransformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_2_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_2/activation_2/mul_3:z:0Stransformer_encoder/transformer/layer_2/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_2/dropout_8/dropout/CastCastJtransformer_encoder/transformer/layer_2/dropout_8/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_2/dropout_8/dropout/CastŠ ?transformer_encoder/transformer/layer_2/dropout_8/dropout/Mul_1MulAtransformer_encoder/transformer/layer_2/dropout_8/dropout/Mul:z:0Btransformer_encoder/transformer/layer_2/dropout_8/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_2/dropout_8/dropout/Mul_1Ć -transformer_encoder/transformer/layer_2/add_1AddV2Ctransformer_encoder/transformer/layer_2/dropout_8/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_2/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_2/add_1ž Xtransformer_encoder/transformer/layer_2/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_2/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_2/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_2/add_1:z:0atransformer_encoder/transformer/layer_2/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_2/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_2/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_2/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_2/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_2/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_2/add_1:z:0Wtransformer_encoder/transformer/layer_2/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_2/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_2/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_2/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_2/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_2/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_2/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_2/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_2/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_2/add_1:z:0Ktransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_2/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_2_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_3_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_3/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_3_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_3/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_3/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_3/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_3/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_3_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_3/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_3_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_3/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_3/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_3/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_3/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_3_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_3/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_3_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_3/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_3/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_3/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_3/self_attention/value/add” Dtransformer_encoder/transformer/layer_3/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_3/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_3/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_3/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_3/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/addŹ Otransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/Softmaxå Ntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2P Ntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Const£ Ltransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/MulMulYtransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/Softmax:softmax:0Wtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2N Ltransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Mul© Ntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/ShapeShapeYtransformer_encoder/transformer/layer_3/self_attention/masked_softmax_3/Softmax:softmax:0* T0* _output_shapes :2P Ntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Shape‘ etransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/random_uniform/RandomUniform RandomUniformWtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02g etransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/random_uniform/RandomUniform÷ Wtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Y Wtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual/yÜ Utransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual GreaterEqualntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/random_uniform/RandomUniform:output:0`transformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2W Utransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/GreaterEqualÜ Mtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/CastCastYtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Cast˜ Ntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Mul_1MulPtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Mul:z:0Qtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Mul_1Ÿ Ftransformer_encoder/transformer/layer_3/self_attention/einsum_1/EinsumEinsumRtransformer_encoder/transformer/layer_3/self_attention/dropout_9/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_3/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_3/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_3_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_3/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_3_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_3/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_3/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_3/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_3/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_3/dropout_10/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_3/dropout_10/dropout/Constć >transformer_encoder/transformer/layer_3/dropout_10/dropout/MulMulOtransformer_encoder/transformer/layer_3/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_3/dropout_10/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_3/dropout_10/dropout/Mulƒ @transformer_encoder/transformer/layer_3/dropout_10/dropout/ShapeShapeOtransformer_encoder/transformer/layer_3/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_3/dropout_10/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_3/dropout_10/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_3/dropout_10/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_3/dropout_10/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_3/dropout_10/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_3/dropout_10/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_3/dropout_10/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_3/dropout_10/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_3/dropout_10/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_3/dropout_10/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_3/dropout_10/dropout/CastCastKtransformer_encoder/transformer/layer_3/dropout_10/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_3/dropout_10/dropout/CastŌ @transformer_encoder/transformer/layer_3/dropout_10/dropout/Mul_1MulBtransformer_encoder/transformer/layer_3/dropout_10/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_3/dropout_10/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_3/dropout_10/dropout/Mul_1ø +transformer_encoder/transformer/layer_3/addAddV2Mtransformer_encoder/transformer/layer_2/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_3/dropout_10/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_3/addŽ `transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_3/add:z:0itransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_3/add:z:0_transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_3/add:z:0Stransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_3/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_3_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_3_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_3/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_3_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_3/intermediate/addAddV2Ktransformer_encoder/transformer/layer_3/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_3/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_3/intermediate/add½ :transformer_encoder/transformer/layer_3/activation_3/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_3/activation_3/Pow/y¾ 8transformer_encoder/transformer/layer_3/activation_3/PowPow transformer_encoder/transformer/layer_3/activation_3/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_3/activation_3/TanhĮ transformer_encoder/transformer/layer_3/activation_3/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_3/activation_3/mul_2½ :transformer_encoder/transformer/layer_3/activation_3/mul_3Multransformer_encoder/transformer/layer_3/activation_3/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_3/activation_3/mul_3± Ktransformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_3_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_3/activation_3/mul_3:z:0Stransformer_encoder/transformer/layer_3/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_3/dropout_11/dropout/MulMul6transformer_encoder/transformer/layer_3/output/add:z:0Itransformer_encoder/transformer/layer_3/dropout_11/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_3/dropout_11/dropout/Mulź @transformer_encoder/transformer/layer_3/dropout_11/dropout/ShapeShape6transformer_encoder/transformer/layer_3/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_3/dropout_11/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_3/dropout_11/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_3/dropout_11/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_3/dropout_11/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_3/dropout_11/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_3/dropout_11/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_3/dropout_11/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_3/dropout_11/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_3/dropout_11/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_3/dropout_11/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_3/dropout_11/dropout/CastCastKtransformer_encoder/transformer/layer_3/dropout_11/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_3/dropout_11/dropout/CastŌ @transformer_encoder/transformer/layer_3/dropout_11/dropout/Mul_1MulBtransformer_encoder/transformer/layer_3/dropout_11/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_3/dropout_11/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_3/dropout_11/dropout/Mul_1Ä -transformer_encoder/transformer/layer_3/add_1AddV2Dtransformer_encoder/transformer/layer_3/dropout_11/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_3/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_3/add_1ž Xtransformer_encoder/transformer/layer_3/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_3/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_3/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_3/add_1:z:0atransformer_encoder/transformer/layer_3/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_3/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_3/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_3/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_3/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_3/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_3/add_1:z:0Wtransformer_encoder/transformer/layer_3/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_3/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_3/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_3/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_3/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_3/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_3/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_3/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_3/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_3/add_1:z:0Ktransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_3/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_3_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_4_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_4/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_4_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_4/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_4/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_4/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_4/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_4_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_4/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_4_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_4/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_4/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_4/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_4/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_4_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_4/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_4_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_4/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_4/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_4/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_4/self_attention/value/add” Dtransformer_encoder/transformer/layer_4/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_4/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_4/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_4/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_4/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/addŹ Otransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/Softmaxē Otransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2Q Otransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Const¦ Mtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/MulMulYtransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/Softmax:softmax:0Xtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Mul« Otransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/ShapeShapeYtransformer_encoder/transformer/layer_4/self_attention/masked_softmax_4/Softmax:softmax:0* T0* _output_shapes :2Q Otransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Shape” ftransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/random_uniform/RandomUniform RandomUniformXtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02h ftransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/random_uniform/RandomUniformł Xtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Z Xtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual/yą Vtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual GreaterEqualotransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/random_uniform/RandomUniform:output:0atransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/GreaterEqualß Ntransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/CastCastZtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Castœ Otransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Mul_1MulQtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Mul:z:0Rtransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Mul_1  Ftransformer_encoder/transformer/layer_4/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_4/self_attention/dropout_12/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_4/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_4/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_4_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_4/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_4_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_4/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_4/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_4/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_4/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_4/dropout_13/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_4/dropout_13/dropout/Constć >transformer_encoder/transformer/layer_4/dropout_13/dropout/MulMulOtransformer_encoder/transformer/layer_4/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_4/dropout_13/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_4/dropout_13/dropout/Mulƒ @transformer_encoder/transformer/layer_4/dropout_13/dropout/ShapeShapeOtransformer_encoder/transformer/layer_4/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_4/dropout_13/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_4/dropout_13/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_4/dropout_13/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_4/dropout_13/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_4/dropout_13/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_4/dropout_13/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_4/dropout_13/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_4/dropout_13/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_4/dropout_13/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_4/dropout_13/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_4/dropout_13/dropout/CastCastKtransformer_encoder/transformer/layer_4/dropout_13/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_4/dropout_13/dropout/CastŌ @transformer_encoder/transformer/layer_4/dropout_13/dropout/Mul_1MulBtransformer_encoder/transformer/layer_4/dropout_13/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_4/dropout_13/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_4/dropout_13/dropout/Mul_1ø +transformer_encoder/transformer/layer_4/addAddV2Mtransformer_encoder/transformer/layer_3/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_4/dropout_13/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_4/addŽ `transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_4/add:z:0itransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_4/add:z:0_transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_4/add:z:0Stransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_4/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_4_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_4_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_4/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_4_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_4/intermediate/addAddV2Ktransformer_encoder/transformer/layer_4/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_4/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_4/intermediate/add½ :transformer_encoder/transformer/layer_4/activation_4/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_4/activation_4/Pow/y¾ 8transformer_encoder/transformer/layer_4/activation_4/PowPow transformer_encoder/transformer/layer_4/activation_4/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_4/activation_4/TanhĮ transformer_encoder/transformer/layer_4/activation_4/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_4/activation_4/mul_2½ :transformer_encoder/transformer/layer_4/activation_4/mul_3Multransformer_encoder/transformer/layer_4/activation_4/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_4/activation_4/mul_3± Ktransformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_4_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_4/activation_4/mul_3:z:0Stransformer_encoder/transformer/layer_4/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_4/dropout_14/dropout/MulMul6transformer_encoder/transformer/layer_4/output/add:z:0Itransformer_encoder/transformer/layer_4/dropout_14/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_4/dropout_14/dropout/Mulź @transformer_encoder/transformer/layer_4/dropout_14/dropout/ShapeShape6transformer_encoder/transformer/layer_4/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_4/dropout_14/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_4/dropout_14/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_4/dropout_14/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_4/dropout_14/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_4/dropout_14/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_4/dropout_14/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_4/dropout_14/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_4/dropout_14/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_4/dropout_14/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_4/dropout_14/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_4/dropout_14/dropout/CastCastKtransformer_encoder/transformer/layer_4/dropout_14/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_4/dropout_14/dropout/CastŌ @transformer_encoder/transformer/layer_4/dropout_14/dropout/Mul_1MulBtransformer_encoder/transformer/layer_4/dropout_14/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_4/dropout_14/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_4/dropout_14/dropout/Mul_1Ä -transformer_encoder/transformer/layer_4/add_1AddV2Dtransformer_encoder/transformer/layer_4/dropout_14/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_4/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_4/add_1ž Xtransformer_encoder/transformer/layer_4/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_4/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_4/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_4/add_1:z:0atransformer_encoder/transformer/layer_4/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_4/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_4/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_4/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_4/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_4/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_4/add_1:z:0Wtransformer_encoder/transformer/layer_4/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_4/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_4/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_4/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_4/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_4/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_4/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_4/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_4/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_4/add_1:z:0Ktransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_4/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_4_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_5_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_5/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_5_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_5/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_5/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_5/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_5/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_5_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_5/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_5_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_5/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_5/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_5/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_5/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_5_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_5/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_5_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_5/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_5/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_5/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_5/self_attention/value/add” Dtransformer_encoder/transformer/layer_5/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_5/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_5/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_5/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_5/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/addŹ Otransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/Softmaxē Otransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2Q Otransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Const¦ Mtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/MulMulYtransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/Softmax:softmax:0Xtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Mul« Otransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/ShapeShapeYtransformer_encoder/transformer/layer_5/self_attention/masked_softmax_5/Softmax:softmax:0* T0* _output_shapes :2Q Otransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Shape” ftransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/random_uniform/RandomUniform RandomUniformXtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02h ftransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/random_uniform/RandomUniformł Xtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Z Xtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual/yą Vtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual GreaterEqualotransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/random_uniform/RandomUniform:output:0atransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/GreaterEqualß Ntransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/CastCastZtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Castœ Otransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Mul_1MulQtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Mul:z:0Rtransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Mul_1  Ftransformer_encoder/transformer/layer_5/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_5/self_attention/dropout_15/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_5/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_5/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_5_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_5/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_5_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_5/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_5/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_5/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_5/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_5/dropout_16/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_5/dropout_16/dropout/Constć >transformer_encoder/transformer/layer_5/dropout_16/dropout/MulMulOtransformer_encoder/transformer/layer_5/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_5/dropout_16/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_5/dropout_16/dropout/Mulƒ @transformer_encoder/transformer/layer_5/dropout_16/dropout/ShapeShapeOtransformer_encoder/transformer/layer_5/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_5/dropout_16/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_5/dropout_16/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_5/dropout_16/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_5/dropout_16/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_5/dropout_16/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_5/dropout_16/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_5/dropout_16/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_5/dropout_16/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_5/dropout_16/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_5/dropout_16/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_5/dropout_16/dropout/CastCastKtransformer_encoder/transformer/layer_5/dropout_16/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_5/dropout_16/dropout/CastŌ @transformer_encoder/transformer/layer_5/dropout_16/dropout/Mul_1MulBtransformer_encoder/transformer/layer_5/dropout_16/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_5/dropout_16/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_5/dropout_16/dropout/Mul_1ø +transformer_encoder/transformer/layer_5/addAddV2Mtransformer_encoder/transformer/layer_4/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_5/dropout_16/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_5/addŽ `transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_5/add:z:0itransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_5/add:z:0_transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_5/add:z:0Stransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_5/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_5_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_5_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_5/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_5_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_5/intermediate/addAddV2Ktransformer_encoder/transformer/layer_5/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_5/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_5/intermediate/add½ :transformer_encoder/transformer/layer_5/activation_5/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_5/activation_5/Pow/y¾ 8transformer_encoder/transformer/layer_5/activation_5/PowPow transformer_encoder/transformer/layer_5/activation_5/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_5/activation_5/TanhĮ transformer_encoder/transformer/layer_5/activation_5/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_5/activation_5/mul_2½ :transformer_encoder/transformer/layer_5/activation_5/mul_3Multransformer_encoder/transformer/layer_5/activation_5/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_5/activation_5/mul_3± Ktransformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_5_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_5/activation_5/mul_3:z:0Stransformer_encoder/transformer/layer_5/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_5/dropout_17/dropout/MulMul6transformer_encoder/transformer/layer_5/output/add:z:0Itransformer_encoder/transformer/layer_5/dropout_17/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_5/dropout_17/dropout/Mulź @transformer_encoder/transformer/layer_5/dropout_17/dropout/ShapeShape6transformer_encoder/transformer/layer_5/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_5/dropout_17/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_5/dropout_17/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_5/dropout_17/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_5/dropout_17/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_5/dropout_17/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_5/dropout_17/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_5/dropout_17/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_5/dropout_17/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_5/dropout_17/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_5/dropout_17/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_5/dropout_17/dropout/CastCastKtransformer_encoder/transformer/layer_5/dropout_17/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_5/dropout_17/dropout/CastŌ @transformer_encoder/transformer/layer_5/dropout_17/dropout/Mul_1MulBtransformer_encoder/transformer/layer_5/dropout_17/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_5/dropout_17/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_5/dropout_17/dropout/Mul_1Ä -transformer_encoder/transformer/layer_5/add_1AddV2Dtransformer_encoder/transformer/layer_5/dropout_17/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_5/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_5/add_1ž Xtransformer_encoder/transformer/layer_5/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_5/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_5/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_5/add_1:z:0atransformer_encoder/transformer/layer_5/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_5/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_5/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_5/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_5/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_5/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_5/add_1:z:0Wtransformer_encoder/transformer/layer_5/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_5/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_5/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_5/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_5/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_5/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_5/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_5/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_5/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_5/add_1:z:0Ktransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_5/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_5_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_6_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_6/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_6_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_6/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_6/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_6/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_6/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_6_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_6/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_6_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_6/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_6/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_6/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_6/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_6_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_6/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_6_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_6/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_6/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_6/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_6/self_attention/value/add” Dtransformer_encoder/transformer/layer_6/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_6/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_6/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_6/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_6/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/addŹ Otransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/Softmaxē Otransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2Q Otransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Const¦ Mtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/MulMulYtransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/Softmax:softmax:0Xtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Mul« Otransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/ShapeShapeYtransformer_encoder/transformer/layer_6/self_attention/masked_softmax_6/Softmax:softmax:0* T0* _output_shapes :2Q Otransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Shape” ftransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/random_uniform/RandomUniform RandomUniformXtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02h ftransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/random_uniform/RandomUniformł Xtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Z Xtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual/yą Vtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual GreaterEqualotransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/random_uniform/RandomUniform:output:0atransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/GreaterEqualß Ntransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/CastCastZtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Castœ Otransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Mul_1MulQtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Mul:z:0Rtransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Mul_1  Ftransformer_encoder/transformer/layer_6/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_6/self_attention/dropout_18/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_6/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_6/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_6_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_6/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_6_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_6/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_6/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_6/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_6/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_6/dropout_19/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_6/dropout_19/dropout/Constć >transformer_encoder/transformer/layer_6/dropout_19/dropout/MulMulOtransformer_encoder/transformer/layer_6/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_6/dropout_19/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_6/dropout_19/dropout/Mulƒ @transformer_encoder/transformer/layer_6/dropout_19/dropout/ShapeShapeOtransformer_encoder/transformer/layer_6/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_6/dropout_19/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_6/dropout_19/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_6/dropout_19/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_6/dropout_19/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_6/dropout_19/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_6/dropout_19/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_6/dropout_19/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_6/dropout_19/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_6/dropout_19/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_6/dropout_19/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_6/dropout_19/dropout/CastCastKtransformer_encoder/transformer/layer_6/dropout_19/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_6/dropout_19/dropout/CastŌ @transformer_encoder/transformer/layer_6/dropout_19/dropout/Mul_1MulBtransformer_encoder/transformer/layer_6/dropout_19/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_6/dropout_19/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_6/dropout_19/dropout/Mul_1ø +transformer_encoder/transformer/layer_6/addAddV2Mtransformer_encoder/transformer/layer_5/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_6/dropout_19/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_6/addŽ `transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_6/add:z:0itransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_6/add:z:0_transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_6/add:z:0Stransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_6/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_6_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_6_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_6/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_6_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_6/intermediate/addAddV2Ktransformer_encoder/transformer/layer_6/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_6/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_6/intermediate/add½ :transformer_encoder/transformer/layer_6/activation_6/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_6/activation_6/Pow/y¾ 8transformer_encoder/transformer/layer_6/activation_6/PowPow transformer_encoder/transformer/layer_6/activation_6/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_6/activation_6/TanhĮ transformer_encoder/transformer/layer_6/activation_6/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_6/activation_6/mul_2½ :transformer_encoder/transformer/layer_6/activation_6/mul_3Multransformer_encoder/transformer/layer_6/activation_6/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_6/activation_6/mul_3± Ktransformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_6_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_6/activation_6/mul_3:z:0Stransformer_encoder/transformer/layer_6/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_6/dropout_20/dropout/MulMul6transformer_encoder/transformer/layer_6/output/add:z:0Itransformer_encoder/transformer/layer_6/dropout_20/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_6/dropout_20/dropout/Mulź @transformer_encoder/transformer/layer_6/dropout_20/dropout/ShapeShape6transformer_encoder/transformer/layer_6/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_6/dropout_20/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_6/dropout_20/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_6/dropout_20/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_6/dropout_20/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_6/dropout_20/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_6/dropout_20/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_6/dropout_20/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_6/dropout_20/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_6/dropout_20/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_6/dropout_20/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_6/dropout_20/dropout/CastCastKtransformer_encoder/transformer/layer_6/dropout_20/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_6/dropout_20/dropout/CastŌ @transformer_encoder/transformer/layer_6/dropout_20/dropout/Mul_1MulBtransformer_encoder/transformer/layer_6/dropout_20/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_6/dropout_20/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_6/dropout_20/dropout/Mul_1Ä -transformer_encoder/transformer/layer_6/add_1AddV2Dtransformer_encoder/transformer/layer_6/dropout_20/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_6/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_6/add_1ž Xtransformer_encoder/transformer/layer_6/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_6/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_6/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_6/add_1:z:0atransformer_encoder/transformer/layer_6/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_6/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_6/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_6/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_6/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_6/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_6/add_1:z:0Wtransformer_encoder/transformer/layer_6/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_6/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_6/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_6/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_6/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_6/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_6/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_6/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_6/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_6/add_1:z:0Ktransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_6/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_6_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_7_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_7/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_7_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_7/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_7/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_7/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_7/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_7_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_7/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_7_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_7/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_7/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_7/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_7/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_7_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_7/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_7_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_7/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_7/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_7/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_7/self_attention/value/add” Dtransformer_encoder/transformer/layer_7/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_7/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_7/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_7/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_7/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/addŹ Otransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/Softmaxē Otransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2Q Otransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Const¦ Mtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/MulMulYtransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/Softmax:softmax:0Xtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Mul« Otransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/ShapeShapeYtransformer_encoder/transformer/layer_7/self_attention/masked_softmax_7/Softmax:softmax:0* T0* _output_shapes :2Q Otransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Shape” ftransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/random_uniform/RandomUniform RandomUniformXtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02h ftransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/random_uniform/RandomUniformł Xtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Z Xtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual/yą Vtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual GreaterEqualotransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/random_uniform/RandomUniform:output:0atransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/GreaterEqualß Ntransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/CastCastZtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Castœ Otransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Mul_1MulQtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Mul:z:0Rtransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Mul_1  Ftransformer_encoder/transformer/layer_7/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_7/self_attention/dropout_21/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_7/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_7/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_7_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_7/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_7_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_7/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_7/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_7/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_7/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_7/dropout_22/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_7/dropout_22/dropout/Constć >transformer_encoder/transformer/layer_7/dropout_22/dropout/MulMulOtransformer_encoder/transformer/layer_7/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_7/dropout_22/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_7/dropout_22/dropout/Mulƒ @transformer_encoder/transformer/layer_7/dropout_22/dropout/ShapeShapeOtransformer_encoder/transformer/layer_7/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_7/dropout_22/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_7/dropout_22/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_7/dropout_22/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_7/dropout_22/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_7/dropout_22/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_7/dropout_22/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_7/dropout_22/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_7/dropout_22/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_7/dropout_22/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_7/dropout_22/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_7/dropout_22/dropout/CastCastKtransformer_encoder/transformer/layer_7/dropout_22/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_7/dropout_22/dropout/CastŌ @transformer_encoder/transformer/layer_7/dropout_22/dropout/Mul_1MulBtransformer_encoder/transformer/layer_7/dropout_22/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_7/dropout_22/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_7/dropout_22/dropout/Mul_1ø +transformer_encoder/transformer/layer_7/addAddV2Mtransformer_encoder/transformer/layer_6/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_7/dropout_22/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_7/addŽ `transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_7/add:z:0itransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_7/add:z:0_transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_7/add:z:0Stransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_7/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_7_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_7_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_7/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_7_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_7/intermediate/addAddV2Ktransformer_encoder/transformer/layer_7/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_7/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_7/intermediate/add½ :transformer_encoder/transformer/layer_7/activation_7/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_7/activation_7/Pow/y¾ 8transformer_encoder/transformer/layer_7/activation_7/PowPow transformer_encoder/transformer/layer_7/activation_7/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_7/activation_7/TanhĮ transformer_encoder/transformer/layer_7/activation_7/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_7/activation_7/mul_2½ :transformer_encoder/transformer/layer_7/activation_7/mul_3Multransformer_encoder/transformer/layer_7/activation_7/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_7/activation_7/mul_3± Ktransformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_7_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_7/activation_7/mul_3:z:0Stransformer_encoder/transformer/layer_7/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_7/dropout_23/dropout/MulMul6transformer_encoder/transformer/layer_7/output/add:z:0Itransformer_encoder/transformer/layer_7/dropout_23/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_7/dropout_23/dropout/Mulź @transformer_encoder/transformer/layer_7/dropout_23/dropout/ShapeShape6transformer_encoder/transformer/layer_7/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_7/dropout_23/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_7/dropout_23/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_7/dropout_23/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_7/dropout_23/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_7/dropout_23/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_7/dropout_23/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_7/dropout_23/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_7/dropout_23/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_7/dropout_23/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_7/dropout_23/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_7/dropout_23/dropout/CastCastKtransformer_encoder/transformer/layer_7/dropout_23/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_7/dropout_23/dropout/CastŌ @transformer_encoder/transformer/layer_7/dropout_23/dropout/Mul_1MulBtransformer_encoder/transformer/layer_7/dropout_23/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_7/dropout_23/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_7/dropout_23/dropout/Mul_1Ä -transformer_encoder/transformer/layer_7/add_1AddV2Dtransformer_encoder/transformer/layer_7/dropout_23/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_7/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_7/add_1ž Xtransformer_encoder/transformer/layer_7/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_7/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_7/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_7/add_1:z:0atransformer_encoder/transformer/layer_7/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_7/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_7/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_7/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_7/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_7/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_7/add_1:z:0Wtransformer_encoder/transformer/layer_7/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_7/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_7/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_7/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_7/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_7/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_7/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_7/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_7/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_7/add_1:z:0Ktransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_7/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_7_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_8_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_8/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_8_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_8/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_8/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_8/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_8/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_8_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_8/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_8_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_8/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_8/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_8/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_8/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_8_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_8/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_8_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_8/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_8/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_8/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_8/self_attention/value/add” Dtransformer_encoder/transformer/layer_8/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_8/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_8/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_8/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_8/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/addŹ Otransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/Softmaxē Otransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2Q Otransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Const¦ Mtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/MulMulYtransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/Softmax:softmax:0Xtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Mul« Otransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/ShapeShapeYtransformer_encoder/transformer/layer_8/self_attention/masked_softmax_8/Softmax:softmax:0* T0* _output_shapes :2Q Otransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Shape” ftransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/random_uniform/RandomUniform RandomUniformXtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02h ftransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/random_uniform/RandomUniformł Xtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Z Xtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual/yą Vtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual GreaterEqualotransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/random_uniform/RandomUniform:output:0atransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/GreaterEqualß Ntransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/CastCastZtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Castœ Otransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Mul_1MulQtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Mul:z:0Rtransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Mul_1  Ftransformer_encoder/transformer/layer_8/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_8/self_attention/dropout_24/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_8/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_8/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_8_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_8/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_8_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_8/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_8/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_8/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_8/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_8/dropout_25/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_8/dropout_25/dropout/Constć >transformer_encoder/transformer/layer_8/dropout_25/dropout/MulMulOtransformer_encoder/transformer/layer_8/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_8/dropout_25/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_8/dropout_25/dropout/Mulƒ @transformer_encoder/transformer/layer_8/dropout_25/dropout/ShapeShapeOtransformer_encoder/transformer/layer_8/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_8/dropout_25/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_8/dropout_25/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_8/dropout_25/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_8/dropout_25/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_8/dropout_25/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_8/dropout_25/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_8/dropout_25/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_8/dropout_25/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_8/dropout_25/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_8/dropout_25/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_8/dropout_25/dropout/CastCastKtransformer_encoder/transformer/layer_8/dropout_25/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_8/dropout_25/dropout/CastŌ @transformer_encoder/transformer/layer_8/dropout_25/dropout/Mul_1MulBtransformer_encoder/transformer/layer_8/dropout_25/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_8/dropout_25/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_8/dropout_25/dropout/Mul_1ø +transformer_encoder/transformer/layer_8/addAddV2Mtransformer_encoder/transformer/layer_7/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_8/dropout_25/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_8/addŽ `transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_8/add:z:0itransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_8/add:z:0_transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_8/add:z:0Stransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_8/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_8_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_8_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_8/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_8_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_8/intermediate/addAddV2Ktransformer_encoder/transformer/layer_8/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_8/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_8/intermediate/add½ :transformer_encoder/transformer/layer_8/activation_8/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_8/activation_8/Pow/y¾ 8transformer_encoder/transformer/layer_8/activation_8/PowPow transformer_encoder/transformer/layer_8/activation_8/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_8/activation_8/TanhĮ transformer_encoder/transformer/layer_8/activation_8/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_8/activation_8/mul_2½ :transformer_encoder/transformer/layer_8/activation_8/mul_3Multransformer_encoder/transformer/layer_8/activation_8/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_8/activation_8/mul_3± Ktransformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_8_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_8/activation_8/mul_3:z:0Stransformer_encoder/transformer/layer_8/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_8/dropout_26/dropout/MulMul6transformer_encoder/transformer/layer_8/output/add:z:0Itransformer_encoder/transformer/layer_8/dropout_26/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_8/dropout_26/dropout/Mulź @transformer_encoder/transformer/layer_8/dropout_26/dropout/ShapeShape6transformer_encoder/transformer/layer_8/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_8/dropout_26/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_8/dropout_26/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_8/dropout_26/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_8/dropout_26/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_8/dropout_26/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_8/dropout_26/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_8/dropout_26/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_8/dropout_26/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_8/dropout_26/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_8/dropout_26/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_8/dropout_26/dropout/CastCastKtransformer_encoder/transformer/layer_8/dropout_26/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_8/dropout_26/dropout/CastŌ @transformer_encoder/transformer/layer_8/dropout_26/dropout/Mul_1MulBtransformer_encoder/transformer/layer_8/dropout_26/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_8/dropout_26/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_8/dropout_26/dropout/Mul_1Ä -transformer_encoder/transformer/layer_8/add_1AddV2Dtransformer_encoder/transformer/layer_8/dropout_26/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_8/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_8/add_1ž Xtransformer_encoder/transformer/layer_8/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_8/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_8/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_8/add_1:z:0atransformer_encoder/transformer/layer_8/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_8/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_8/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_8/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_8/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_8/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_8/add_1:z:0Wtransformer_encoder/transformer/layer_8/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_8/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_8/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_8/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_8/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_8/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_8/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_8/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_8/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_8/add_1:z:0Ktransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_8/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_8_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1Ž Ytransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_9_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_9/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum» Otransformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_9_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOpž @transformer_encoder/transformer/layer_9/self_attention/query/addAddV2Stransformer_encoder/transformer/layer_9/self_attention/query/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_9/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_9/self_attention/query/addŲ Wtransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_9_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Y Wtransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp· Htransformer_encoder/transformer/layer_9/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0_transformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2J Htransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsumµ Mtransformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOpReadVariableOpVtransformer_encoder_transformer_layer_9_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02O Mtransformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOpö >transformer_encoder/transformer/layer_9/self_attention/key/addAddV2Qtransformer_encoder/transformer/layer_9/self_attention/key/einsum/Einsum:output:0Utransformer_encoder/transformer/layer_9/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2@ >transformer_encoder/transformer/layer_9/self_attention/key/addŽ Ytransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpbtransformer_encoder_transformer_layer_9_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02[ Ytransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp½ Jtransformer_encoder/transformer/layer_9/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0atransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2L Jtransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum» Otransformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOpReadVariableOpXtransformer_encoder_transformer_layer_9_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02Q Otransformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOpž @transformer_encoder/transformer/layer_9/self_attention/value/addAddV2Stransformer_encoder/transformer/layer_9/self_attention/value/einsum/Einsum:output:0Wtransformer_encoder/transformer/layer_9/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2B @transformer_encoder/transformer/layer_9/self_attention/value/add” Dtransformer_encoder/transformer/layer_9/self_attention/einsum/EinsumEinsumBtransformer_encoder/transformer/layer_9/self_attention/key/add:z:0Dtransformer_encoder/transformer/layer_9/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2F Dtransformer_encoder/transformer/layer_9/self_attention/einsum/EinsumĮ 2> transformer_encoder/transformer/layer_9/self_attention/Mul:z:0Otransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2M Ktransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/addŹ Otransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/SoftmaxSoftmaxOtransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/Softmaxē Otransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2Q Otransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Const¦ Mtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/MulMulYtransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/Softmax:softmax:0Xtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Mul« Otransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/ShapeShapeYtransformer_encoder/transformer/layer_9/self_attention/masked_softmax_9/Softmax:softmax:0* T0* _output_shapes :2Q Otransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Shape” ftransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/random_uniform/RandomUniform RandomUniformXtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02h ftransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/random_uniform/RandomUniformł Xtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2Z Xtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual/yą Vtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual GreaterEqualotransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/random_uniform/RandomUniform:output:0atransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/GreaterEqualß Ntransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/CastCastZtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Castœ Otransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Mul_1MulQtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Mul:z:0Rtransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Mul_1  Ftransformer_encoder/transformer/layer_9/self_attention/einsum_1/EinsumEinsumStransformer_encoder/transformer/layer_9/self_attention/dropout_27/dropout/Mul_1:z:0Dtransformer_encoder/transformer/layer_9/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2H Ftransformer_encoder/transformer/layer_9/self_attention/einsum_1/Einsum’ dtransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpmtransformer_encoder_transformer_layer_9_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02f dtransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOpŻ Utransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/EinsumEinsumOtransformer_encoder/transformer/layer_9/self_attention/einsum_1/Einsum:output:0ltransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2W Utransformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/EinsumŁ Ztransformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_9_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOp§ Ktransformer_encoder/transformer/layer_9/self_attention/attention_output/addAddV2^transformer_encoder/transformer/layer_9/self_attention/attention_output/einsum/Einsum:output:0btransformer_encoder/transformer/layer_9/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2M Ktransformer_encoder/transformer/layer_9/self_attention/attention_output/addÉ @transformer_encoder/transformer/layer_9/dropout_28/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2B @transformer_encoder/transformer/layer_9/dropout_28/dropout/Constć >transformer_encoder/transformer/layer_9/dropout_28/dropout/MulMulOtransformer_encoder/transformer/layer_9/self_attention/attention_output/add:z:0Itransformer_encoder/transformer/layer_9/dropout_28/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_9/dropout_28/dropout/Mulƒ @transformer_encoder/transformer/layer_9/dropout_28/dropout/ShapeShapeOtransformer_encoder/transformer/layer_9/self_attention/attention_output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_9/dropout_28/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_9/dropout_28/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_9/dropout_28/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_9/dropout_28/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_9/dropout_28/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_9/dropout_28/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_9/dropout_28/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_9/dropout_28/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_9/dropout_28/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_9/dropout_28/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_9/dropout_28/dropout/CastCastKtransformer_encoder/transformer/layer_9/dropout_28/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_9/dropout_28/dropout/CastŌ @transformer_encoder/transformer/layer_9/dropout_28/dropout/Mul_1MulBtransformer_encoder/transformer/layer_9/dropout_28/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_9/dropout_28/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_9/dropout_28/dropout/Mul_1ø +transformer_encoder/transformer/layer_9/addAddV2Mtransformer_encoder/transformer/layer_8/output_layer_norm/batchnorm/add_1:z:0Dtransformer_encoder/transformer/layer_9/dropout_28/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2- +transformer_encoder/transformer/layer_9/addŽ `transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2b `transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indices” Ntransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/meanMean/transformer_encoder/transformer/layer_9/add:z:0itransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2P Ntransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/meanŲ Vtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/StopGradient StopGradientWtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2X Vtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/StopGradient” [transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference/transformer_encoder/transformer/layer_9/add:z:0_transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2] [transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/SquaredDifference– dtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2f dtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indicesŠ Rtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/varianceMean_transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/SquaredDifference:z:0mtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2T Rtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/varianceė Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add/y£ Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/addAddV2[transformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/variance:output:0Ztransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/addĆ Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/RsqrtRsqrtStransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrtå ^transformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOpgtransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02` ^transformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOpØ Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mulMulUtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/Rsqrt:y:0ftransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/muló Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1Mul/transformer_encoder/transformer/layer_9/add:z:0Stransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1› Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2MulWtransformer_encoder/transformer/layer_9/self_attention_layer_norm/moments/mean:output:0Stransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2Ł Ztransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_9_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02\ Ztransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOp¤ Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/subSubbtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Utransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2Q Otransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/sub› Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1AddV2Utransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/mul_1:z:0Stransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2S Qtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1Ć Qtransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpReadVariableOpZtransformer_encoder_transformer_layer_9_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02S Qtransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOpØ Btransformer_encoder/transformer/layer_9/intermediate/einsum/EinsumEinsumUtransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0Ytransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2D Btransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum  Gtransformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOpReadVariableOpPtransformer_encoder_transformer_layer_9_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02I Gtransformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOpŪ 8transformer_encoder/transformer/layer_9/intermediate/addAddV2Ktransformer_encoder/transformer/layer_9/intermediate/einsum/Einsum:output:0Otransformer_encoder/transformer/layer_9/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2: 8transformer_encoder/transformer/layer_9/intermediate/add½ :transformer_encoder/transformer/layer_9/activation_9/Pow/yConst* _output_shapes :* dtype0* value B *@@2< :transformer_encoder/transformer/layer_9/activation_9/Pow/y¾ 8transformer_encoder/transformer/layer_9/activation_9/PowPow transformer_encoder/transformer/layer_9/activation_9/mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_9/activation_9/TanhĮ transformer_encoder/transformer/layer_9/activation_9/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_9/activation_9/mul_2½ :transformer_encoder/transformer/layer_9/activation_9/mul_3Multransformer_encoder/transformer/layer_9/activation_9/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_9/activation_9/mul_3± Ktransformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOpReadVariableOpTtransformer_encoder_transformer_layer_9_output_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02M Ktransformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOp’ transformer_encoder/transformer/layer_9/activation_9/mul_3:z:0Stransformer_encoder/transformer/layer_9/output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2> transformer_encoder/transformer/layer_9/dropout_29/dropout/MulMul6transformer_encoder/transformer/layer_9/output/add:z:0Itransformer_encoder/transformer/layer_9/dropout_29/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2@ >transformer_encoder/transformer/layer_9/dropout_29/dropout/Mulź @transformer_encoder/transformer/layer_9/dropout_29/dropout/ShapeShape6transformer_encoder/transformer/layer_9/output/add:z:0* T0* _output_shapes :2B @transformer_encoder/transformer/layer_9/dropout_29/dropout/ShapeŪ Wtransformer_encoder/transformer/layer_9/dropout_29/dropout/random_uniform/RandomUniform RandomUniformItransformer_encoder/transformer/layer_9/dropout_29/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Y Wtransformer_encoder/transformer/layer_9/dropout_29/dropout/random_uniform/RandomUniformŪ Itransformer_encoder/transformer/layer_9/dropout_29/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2K Itransformer_encoder/transformer/layer_9/dropout_29/dropout/GreaterEqual/y˜ Gtransformer_encoder/transformer/layer_9/dropout_29/dropout/GreaterEqual GreaterEqual`transformer_encoder/transformer/layer_9/dropout_29/dropout/random_uniform/RandomUniform:output:0Rtransformer_encoder/transformer/layer_9/dropout_29/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_9/dropout_29/dropout/GreaterEqual¦ ?transformer_encoder/transformer/layer_9/dropout_29/dropout/CastCastKtransformer_encoder/transformer/layer_9/dropout_29/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_9/dropout_29/dropout/CastŌ @transformer_encoder/transformer/layer_9/dropout_29/dropout/Mul_1MulBtransformer_encoder/transformer/layer_9/dropout_29/dropout/Mul:z:0Ctransformer_encoder/transformer/layer_9/dropout_29/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_9/dropout_29/dropout/Mul_1Ä -transformer_encoder/transformer/layer_9/add_1AddV2Dtransformer_encoder/transformer/layer_9/dropout_29/dropout/Mul_1:z:0Utransformer_encoder/transformer/layer_9/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2/ -transformer_encoder/transformer/layer_9/add_1ž Xtransformer_encoder/transformer/layer_9/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_9/output_layer_norm/moments/mean/reduction_indicesž Ftransformer_encoder/transformer/layer_9/output_layer_norm/moments/meanMean1transformer_encoder/transformer/layer_9/add_1:z:0atransformer_encoder/transformer/layer_9/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2H Ftransformer_encoder/transformer/layer_9/output_layer_norm/moments/meanĄ Ntransformer_encoder/transformer/layer_9/output_layer_norm/moments/StopGradient StopGradientOtransformer_encoder/transformer/layer_9/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_9/output_layer_norm/moments/StopGradient‹ Stransformer_encoder/transformer/layer_9/output_layer_norm/moments/SquaredDifferenceSquaredDifference1transformer_encoder/transformer/layer_9/add_1:z:0Wtransformer_encoder/transformer/layer_9/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2U Stransformer_encoder/transformer/layer_9/output_layer_norm/moments/SquaredDifference† \transformer_encoder/transformer/layer_9/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2^ \transformer_encoder/transformer/layer_9/output_layer_norm/moments/variance/reduction_indices° Jtransformer_encoder/transformer/layer_9/output_layer_norm/moments/varianceMeanWtransformer_encoder/transformer/layer_9/output_layer_norm/moments/SquaredDifference:z:0etransformer_encoder/transformer/layer_9/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2L Jtransformer_encoder/transformer/layer_9/output_layer_norm/moments/varianceŪ Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2K Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add/yƒ Gtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/addAddV2Stransformer_encoder/transformer/layer_9/output_layer_norm/moments/variance:output:0Rtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2I Gtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add« Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/RsqrtRsqrtKtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2K Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/RsqrtĶ Vtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp_transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02X Vtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOpˆ Gtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mulMulMtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/Rsqrt:y:0^transformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mulŻ Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_1Mul1transformer_encoder/transformer/layer_9/add_1:z:0Ktransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_1ū Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_2MulOtransformer_encoder/transformer/layer_9/output_layer_norm/moments/mean:output:0Ktransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_2Į Rtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_9_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02T Rtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp„ Gtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/subSubZtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/ReadVariableOp:value:0Mtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2I Gtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/subū Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1AddV2Mtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/mul_1:z:0Ktransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2K Itransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1į Ztransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_10_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02\ Ztransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOpĄ Ktransformer_encoder/transformer/layer_10/self_attention/query/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0btransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2M Ktransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_10_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_10/self_attention/query/addAddV2Ttransformer_encoder/transformer/layer_10/self_attention/query/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_10/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_10/self_attention/query/addŪ Xtransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpatransformer_encoder_transformer_layer_10_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Z Xtransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOpŗ Itransformer_encoder/transformer/layer_10/self_attention/key/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0`transformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2K Itransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsumø Ntransformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOpReadVariableOpWtransformer_encoder_transformer_layer_10_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02P Ntransformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOpś ?transformer_encoder/transformer/layer_10/self_attention/key/addAddV2Rtransformer_encoder/transformer/layer_10/self_attention/key/einsum/Einsum:output:0Vtransformer_encoder/transformer/layer_10/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2A ?transformer_encoder/transformer/layer_10/self_attention/key/addį Ztransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_10_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02\ Ztransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOpĄ Ktransformer_encoder/transformer/layer_10/self_attention/value/einsum/EinsumEinsumMtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0btransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2M Ktransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_10_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_10/self_attention/value/addAddV2Ttransformer_encoder/transformer/layer_10/self_attention/value/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_10/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_10/self_attention/value/add˜ Etransformer_encoder/transformer/layer_10/self_attention/einsum/EinsumEinsumCtransformer_encoder/transformer/layer_10/self_attention/key/add:z:0Etransformer_encoder/transformer/layer_10/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2G Etransformer_encoder/transformer/layer_10/self_attention/einsum/EinsumĆ =transformer_encoder/transformer/layer_10/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2? =transformer_encoder/transformer/layer_10/self_attention/Mul/yå ;transformer_encoder/transformer/layer_10/self_attention/MulMulNtransformer_encoder/transformer/layer_10/self_attention/einsum/Einsum:output:0Ftransformer_encoder/transformer/layer_10/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer_encoder/transformer/layer_10/self_attention/Mulž Xtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dimš Ttransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims ExpandDims/transformer_encoder/self_attention_mask/mul:z:0atransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2V Ttransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDimsē Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub/xConst* _output_shapes :* dtype0* value B *€?2Q Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub/xŖ Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/subSubXtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub/x:output:0]transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/subē Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul/yConst* _output_shapes :* dtype0* value B *@Ę2Q Otransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul/yž Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mulMulQtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/sub:z:0Xtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul‡ Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/addAddV2?transformer_encoder/transformer/layer_10/self_attention/Mul:z:0Qtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/addŠ Qtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/SoftmaxSoftmaxQtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/Softmaxé Ptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2R Ptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Const« Ntransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/MulMul[transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/Softmax:softmax:0Ytransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/MulÆ Ptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/ShapeShape[transformer_encoder/transformer/layer_10/self_attention/masked_softmax_10/Softmax:softmax:0* T0* _output_shapes :2R Ptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Shape— gtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/random_uniform/RandomUniform RandomUniformYtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02i gtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/random_uniform/RandomUniformū Ytransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2[ Ytransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/GreaterEqual/yä Wtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/GreaterEqual GreaterEqualptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/random_uniform/RandomUniform:output:0btransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Y Wtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/GreaterEqualā Otransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/CastCast[transformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Cast  Ptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Mul_1MulRtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Mul:z:0Stransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2R Ptransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Mul_1¤ Gtransformer_encoder/transformer/layer_10/self_attention/einsum_1/EinsumEinsumTtransformer_encoder/transformer/layer_10/self_attention/dropout_30/dropout/Mul_1:z:0Etransformer_encoder/transformer/layer_10/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2I Gtransformer_encoder/transformer/layer_10/self_attention/einsum_1/Einsum‚ etransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpntransformer_encoder_transformer_layer_10_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02g etransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOpį Vtransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/EinsumEinsumPtransformer_encoder/transformer/layer_10/self_attention/einsum_1/Einsum:output:0mtransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2X Vtransformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/EinsumÜ [transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOpReadVariableOpdtransformer_encoder_transformer_layer_10_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02] [transformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOp« Ltransformer_encoder/transformer/layer_10/self_attention/attention_output/addAddV2_transformer_encoder/transformer/layer_10/self_attention/attention_output/einsum/Einsum:output:0ctransformer_encoder/transformer/layer_10/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2N Ltransformer_encoder/transformer/layer_10/self_attention/attention_output/addĖ Atransformer_encoder/transformer/layer_10/dropout_31/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2C Atransformer_encoder/transformer/layer_10/dropout_31/dropout/Constē ?transformer_encoder/transformer/layer_10/dropout_31/dropout/MulMulPtransformer_encoder/transformer/layer_10/self_attention/attention_output/add:z:0Jtransformer_encoder/transformer/layer_10/dropout_31/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_10/dropout_31/dropout/Mul† Atransformer_encoder/transformer/layer_10/dropout_31/dropout/ShapeShapePtransformer_encoder/transformer/layer_10/self_attention/attention_output/add:z:0* T0* _output_shapes :2C Atransformer_encoder/transformer/layer_10/dropout_31/dropout/ShapeŽ Xtransformer_encoder/transformer/layer_10/dropout_31/dropout/random_uniform/RandomUniform RandomUniformJtransformer_encoder/transformer/layer_10/dropout_31/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Z Xtransformer_encoder/transformer/layer_10/dropout_31/dropout/random_uniform/RandomUniformŻ Jtransformer_encoder/transformer/layer_10/dropout_31/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2L Jtransformer_encoder/transformer/layer_10/dropout_31/dropout/GreaterEqual/yœ Htransformer_encoder/transformer/layer_10/dropout_31/dropout/GreaterEqual GreaterEqualatransformer_encoder/transformer/layer_10/dropout_31/dropout/random_uniform/RandomUniform:output:0Stransformer_encoder/transformer/layer_10/dropout_31/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_10/dropout_31/dropout/GreaterEqual© @transformer_encoder/transformer/layer_10/dropout_31/dropout/CastCastLtransformer_encoder/transformer/layer_10/dropout_31/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_10/dropout_31/dropout/CastŲ Atransformer_encoder/transformer/layer_10/dropout_31/dropout/Mul_1MulCtransformer_encoder/transformer/layer_10/dropout_31/dropout/Mul:z:0Dtransformer_encoder/transformer/layer_10/dropout_31/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2C Atransformer_encoder/transformer/layer_10/dropout_31/dropout/Mul_1» ,transformer_encoder/transformer/layer_10/addAddV2Mtransformer_encoder/transformer/layer_9/output_layer_norm/batchnorm/add_1:z:0Etransformer_encoder/transformer/layer_10/dropout_31/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer_encoder/transformer/layer_10/add atransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2c atransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean/reduction_indices˜ Otransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/meanMean0transformer_encoder/transformer/layer_10/add:z:0jtransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Q Otransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/meanŪ Wtransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/StopGradient StopGradientXtransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Y Wtransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/StopGradient„ \transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference0transformer_encoder/transformer/layer_10/add:z:0`transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2^ \transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/SquaredDifference˜ etransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2g etransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance/reduction_indicesŌ Stransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/varianceMean`transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/SquaredDifference:z:0ntransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Stransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/varianceķ Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2T Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add/y§ Ptransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/addAddV2\transformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/variance:output:0[transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2R Ptransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/addĘ Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/RsqrtRsqrtTtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2T Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/Rsqrtč _transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOphtransformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02a _transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOp¬ Ptransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mulMulVtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/Rsqrt:y:0gtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2R Ptransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul÷ Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Mul0transformer_encoder/transformer/layer_10/add:z:0Ttransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2T Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1Ÿ Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2MulXtransformer_encoder/transformer/layer_10/self_attention_layer_norm/moments/mean:output:0Ttransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2T Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2Ü [transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpdtransformer_encoder_transformer_layer_10_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02] [transformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOpØ Ptransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/subSubctransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Vtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2R Ptransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/subŸ Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1AddV2Vtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/mul_1:z:0Ttransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2T Rtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1Ę Rtransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_10_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02T Rtransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOp¬ Ctransformer_encoder/transformer/layer_10/intermediate/einsum/EinsumEinsumVtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0Ztransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2E Ctransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum£ Htransformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOpReadVariableOpQtransformer_encoder_transformer_layer_10_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02J Htransformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOpß 9transformer_encoder/transformer/layer_10/intermediate/addAddV2Ltransformer_encoder/transformer/layer_10/intermediate/einsum/Einsum:output:0Ptransformer_encoder/transformer/layer_10/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_10/intermediate/addĮ transformer_encoder/transformer/layer_10/activation_10/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_10/activation_10/mulĄ :transformer_encoder/transformer/layer_10/activation_10/addAddV2=transformer_encoder/transformer/layer_10/intermediate/add:z:0>transformer_encoder/transformer/layer_10/activation_10/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_10/activation_10/addÅ >transformer_encoder/transformer/layer_10/activation_10/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2@ >transformer_encoder/transformer/layer_10/activation_10/mul_1/xĢ transformer_encoder/transformer/layer_10/activation_10/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2> transformer_encoder/transformer/layer_10/activation_10/add_1/xConst* _output_shapes :* dtype0* value B *€?2@ >transformer_encoder/transformer/layer_10/activation_10/add_1/xĻ transformer_encoder/transformer/layer_10/activation_10/mul_2/xConst* _output_shapes :* dtype0* value B *?2@ >transformer_encoder/transformer/layer_10/activation_10/mul_2/xĪ abd2? =transformer_encoder/transformer/layer_10/output/einsum/Einsum‘ Btransformer_encoder/transformer/layer_10/output/add/ReadVariableOpReadVariableOpKtransformer_encoder_transformer_layer_10_output_add_readvariableop_resource* _output_shapes :€* dtype02D Btransformer_encoder/transformer/layer_10/output/add/ReadVariableOpĒ 3transformer_encoder/transformer/layer_10/output/addAddV2Ftransformer_encoder/transformer/layer_10/output/einsum/Einsum:output:0Jtransformer_encoder/transformer/layer_10/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer_encoder/transformer/layer_10/output/addĖ Atransformer_encoder/transformer/layer_10/dropout_32/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2C Atransformer_encoder/transformer/layer_10/dropout_32/dropout/ConstĪ ?transformer_encoder/transformer/layer_10/dropout_32/dropout/MulMul7transformer_encoder/transformer/layer_10/output/add:z:0Jtransformer_encoder/transformer/layer_10/dropout_32/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_10/dropout_32/dropout/Mulķ Atransformer_encoder/transformer/layer_10/dropout_32/dropout/ShapeShape7transformer_encoder/transformer/layer_10/output/add:z:0* T0* _output_shapes :2C Atransformer_encoder/transformer/layer_10/dropout_32/dropout/ShapeŽ Xtransformer_encoder/transformer/layer_10/dropout_32/dropout/random_uniform/RandomUniform RandomUniformJtransformer_encoder/transformer/layer_10/dropout_32/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Z Xtransformer_encoder/transformer/layer_10/dropout_32/dropout/random_uniform/RandomUniformŻ Jtransformer_encoder/transformer/layer_10/dropout_32/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2L Jtransformer_encoder/transformer/layer_10/dropout_32/dropout/GreaterEqual/yœ Htransformer_encoder/transformer/layer_10/dropout_32/dropout/GreaterEqual GreaterEqualatransformer_encoder/transformer/layer_10/dropout_32/dropout/random_uniform/RandomUniform:output:0Stransformer_encoder/transformer/layer_10/dropout_32/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_10/dropout_32/dropout/GreaterEqual© @transformer_encoder/transformer/layer_10/dropout_32/dropout/CastCastLtransformer_encoder/transformer/layer_10/dropout_32/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_10/dropout_32/dropout/CastŲ Atransformer_encoder/transformer/layer_10/dropout_32/dropout/Mul_1MulCtransformer_encoder/transformer/layer_10/dropout_32/dropout/Mul:z:0Dtransformer_encoder/transformer/layer_10/dropout_32/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2C Atransformer_encoder/transformer/layer_10/dropout_32/dropout/Mul_1Č .transformer_encoder/transformer/layer_10/add_1AddV2Etransformer_encoder/transformer/layer_10/dropout_32/dropout/Mul_1:z:0Vtransformer_encoder/transformer/layer_10/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€20 .transformer_encoder/transformer/layer_10/add_1€ Ytransformer_encoder/transformer/layer_10/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2[ Ytransformer_encoder/transformer/layer_10/output_layer_norm/moments/mean/reduction_indices‚ Gtransformer_encoder/transformer/layer_10/output_layer_norm/moments/meanMean2transformer_encoder/transformer/layer_10/add_1:z:0btransformer_encoder/transformer/layer_10/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2I Gtransformer_encoder/transformer/layer_10/output_layer_norm/moments/meanĆ Otransformer_encoder/transformer/layer_10/output_layer_norm/moments/StopGradient StopGradientPtransformer_encoder/transformer/layer_10/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_10/output_layer_norm/moments/StopGradient Ttransformer_encoder/transformer/layer_10/output_layer_norm/moments/SquaredDifferenceSquaredDifference2transformer_encoder/transformer/layer_10/add_1:z:0Xtransformer_encoder/transformer/layer_10/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Ttransformer_encoder/transformer/layer_10/output_layer_norm/moments/SquaredDifferenceˆ ]transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2_ ]transformer_encoder/transformer/layer_10/output_layer_norm/moments/variance/reduction_indices“ Ktransformer_encoder/transformer/layer_10/output_layer_norm/moments/varianceMeanXtransformer_encoder/transformer/layer_10/output_layer_norm/moments/SquaredDifference:z:0ftransformer_encoder/transformer/layer_10/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2M Ktransformer_encoder/transformer/layer_10/output_layer_norm/moments/varianceŻ Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2L Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add/y‡ Htransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/addAddV2Ttransformer_encoder/transformer/layer_10/output_layer_norm/moments/variance:output:0Stransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2J Htransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add® Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/RsqrtRsqrtLtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/RsqrtŠ Wtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02Y Wtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul/ReadVariableOpŒ Htransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mulMulNtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/Rsqrt:y:0_transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mulį Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_1Mul2transformer_encoder/transformer/layer_10/add_1:z:0Ltransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2L Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_1’ Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_2MulPtransformer_encoder/transformer/layer_10/output_layer_norm/moments/mean:output:0Ltransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2L Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_2Ä Stransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp\transformer_encoder_transformer_layer_10_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02U Stransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/ReadVariableOpˆ Htransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/subSub[transformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/ReadVariableOp:value:0Ntransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/sub’ Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1AddV2Ntransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/mul_1:z:0Ltransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2L Jtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1į Ztransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_11_self_attention_query_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02\ Ztransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum/ReadVariableOpĮ Ktransformer_encoder/transformer/layer_11/self_attention/query/einsum/EinsumEinsumNtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0btransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2M Ktransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_11_self_attention_query_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_11/self_attention/query/addAddV2Ttransformer_encoder/transformer/layer_11/self_attention/query/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_11/self_attention/query/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_11/self_attention/query/addŪ Xtransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOpReadVariableOpatransformer_encoder_transformer_layer_11_self_attention_key_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02Z Xtransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOp» Itransformer_encoder/transformer/layer_11/self_attention/key/einsum/EinsumEinsumNtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0`transformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2K Itransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsumø Ntransformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOpReadVariableOpWtransformer_encoder_transformer_layer_11_self_attention_key_add_readvariableop_resource* _output_shapes : @* dtype02P Ntransformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOpś ?transformer_encoder/transformer/layer_11/self_attention/key/addAddV2Rtransformer_encoder/transformer/layer_11/self_attention/key/einsum/Einsum:output:0Vtransformer_encoder/transformer/layer_11/self_attention/key/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2A ?transformer_encoder/transformer/layer_11/self_attention/key/addį Ztransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpReadVariableOpctransformer_encoder_transformer_layer_11_self_attention_value_einsum_einsum_readvariableop_resource*# _output_shapes : € @* dtype02\ Ztransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOpĮ Ktransformer_encoder/transformer/layer_11/self_attention/value/einsum/EinsumEinsumNtransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0btransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum/ReadVariableOp:value:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equation abc,cde->abde2M Ktransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum¾ Ptransformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOpReadVariableOpYtransformer_encoder_transformer_layer_11_self_attention_value_add_readvariableop_resource* _output_shapes : @* dtype02R Ptransformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOp‚ Atransformer_encoder/transformer/layer_11/self_attention/value/addAddV2Ttransformer_encoder/transformer/layer_11/self_attention/value/einsum/Einsum:output:0Xtransformer_encoder/transformer/layer_11/self_attention/value/add/ReadVariableOp:value:0* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @2C Atransformer_encoder/transformer/layer_11/self_attention/value/add˜ Etransformer_encoder/transformer/layer_11/self_attention/einsum/EinsumEinsumCtransformer_encoder/transformer/layer_11/self_attention/key/add:z:0Etransformer_encoder/transformer/layer_11/self_attention/query/add:z:0* N* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* equationaecd,abcd->acbe2G Etransformer_encoder/transformer/layer_11/self_attention/einsum/EinsumĆ =transformer_encoder/transformer/layer_11/self_attention/Mul/yConst* _output_shapes :* dtype0* value B *>2? =transformer_encoder/transformer/layer_11/self_attention/Mul/yå ;transformer_encoder/transformer/layer_11/self_attention/MulMulNtransformer_encoder/transformer/layer_11/self_attention/einsum/Einsum:output:0Ftransformer_encoder/transformer/layer_11/self_attention/Mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2= ;transformer_encoder/transformer/layer_11/self_attention/Mulž Xtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimConst* _output_shapes :* dtype0* value B :2Z Xtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dimš Ttransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims ExpandDims/transformer_encoder/self_attention_mask/mul:z:0atransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims/dim:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2V Ttransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDimsē Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub/xConst* _output_shapes :* dtype0* value B *€?2Q Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub/xŖ Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/subSubXtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub/x:output:0]transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/ExpandDims:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/subē Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul/yConst* _output_shapes :* dtype0* value B *@Ę2Q Otransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul/yž Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mulMulQtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/sub:z:0Xtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul‡ Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/addAddV2?transformer_encoder/transformer/layer_11/self_attention/Mul:z:0Qtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/mul:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2O Mtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/addŠ Qtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/SoftmaxSoftmaxQtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/add:z:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2S Qtransformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/Softmaxé Ptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2R Ptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Const« Ntransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/MulMul[transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/Softmax:softmax:0Ytransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Const:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2P Ntransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/MulÆ Ptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/ShapeShape[transformer_encoder/transformer/layer_11/self_attention/masked_softmax_11/Softmax:softmax:0* T0* _output_shapes :2R Ptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Shape— gtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/random_uniform/RandomUniform RandomUniformYtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Shape:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’* dtype02i gtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/random_uniform/RandomUniformū Ytransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2[ Ytransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/GreaterEqual/yä Wtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/GreaterEqual GreaterEqualptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/random_uniform/RandomUniform:output:0btransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/GreaterEqual/y:output:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Y Wtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/GreaterEqualā Otransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/CastCast[transformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/GreaterEqual:z:0* DstT0* SrcT0 *A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Cast  Ptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Mul_1MulRtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Mul:z:0Stransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Cast:y:0* T0*A _output_shapes/ -:+ ’’’’’’’’’  ’’’’’’’’’ ’’’’’’’’’2R Ptransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Mul_1¤ Gtransformer_encoder/transformer/layer_11/self_attention/einsum_1/EinsumEinsumTtransformer_encoder/transformer/layer_11/self_attention/dropout_33/dropout/Mul_1:z:0Etransformer_encoder/transformer/layer_11/self_attention/value/add:z:0* N* T0*8 _output_shapes& $:" ’’’’’’’’’ ’’’’’’’’’ @* equationacbe,aecd->abcd2I Gtransformer_encoder/transformer/layer_11/self_attention/einsum_1/Einsum‚ etransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpReadVariableOpntransformer_encoder_transformer_layer_11_self_attention_attention_output_einsum_einsum_readvariableop_resource*# _output_shapes :  @€* dtype02g etransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOpį Vtransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/EinsumEinsumPtransformer_encoder/transformer/layer_11/self_attention/einsum_1/Einsum:output:0mtransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation abcd,cde->abe2X Vtransformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/EinsumÜ [transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOpReadVariableOpdtransformer_encoder_transformer_layer_11_self_attention_attention_output_add_readvariableop_resource* _output_shapes :€* dtype02] [transformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOp« Ltransformer_encoder/transformer/layer_11/self_attention/attention_output/addAddV2_transformer_encoder/transformer/layer_11/self_attention/attention_output/einsum/Einsum:output:0ctransformer_encoder/transformer/layer_11/self_attention/attention_output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2N Ltransformer_encoder/transformer/layer_11/self_attention/attention_output/addĖ Atransformer_encoder/transformer/layer_11/dropout_34/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2C Atransformer_encoder/transformer/layer_11/dropout_34/dropout/Constē ?transformer_encoder/transformer/layer_11/dropout_34/dropout/MulMulPtransformer_encoder/transformer/layer_11/self_attention/attention_output/add:z:0Jtransformer_encoder/transformer/layer_11/dropout_34/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_11/dropout_34/dropout/Mul† Atransformer_encoder/transformer/layer_11/dropout_34/dropout/ShapeShapePtransformer_encoder/transformer/layer_11/self_attention/attention_output/add:z:0* T0* _output_shapes :2C Atransformer_encoder/transformer/layer_11/dropout_34/dropout/ShapeŽ Xtransformer_encoder/transformer/layer_11/dropout_34/dropout/random_uniform/RandomUniform RandomUniformJtransformer_encoder/transformer/layer_11/dropout_34/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Z Xtransformer_encoder/transformer/layer_11/dropout_34/dropout/random_uniform/RandomUniformŻ Jtransformer_encoder/transformer/layer_11/dropout_34/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2L Jtransformer_encoder/transformer/layer_11/dropout_34/dropout/GreaterEqual/yœ Htransformer_encoder/transformer/layer_11/dropout_34/dropout/GreaterEqual GreaterEqualatransformer_encoder/transformer/layer_11/dropout_34/dropout/random_uniform/RandomUniform:output:0Stransformer_encoder/transformer/layer_11/dropout_34/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_11/dropout_34/dropout/GreaterEqual© @transformer_encoder/transformer/layer_11/dropout_34/dropout/CastCastLtransformer_encoder/transformer/layer_11/dropout_34/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_11/dropout_34/dropout/CastŲ Atransformer_encoder/transformer/layer_11/dropout_34/dropout/Mul_1MulCtransformer_encoder/transformer/layer_11/dropout_34/dropout/Mul:z:0Dtransformer_encoder/transformer/layer_11/dropout_34/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2C Atransformer_encoder/transformer/layer_11/dropout_34/dropout/Mul_1¼ ,transformer_encoder/transformer/layer_11/addAddV2Ntransformer_encoder/transformer/layer_10/output_layer_norm/batchnorm/add_1:z:0Etransformer_encoder/transformer/layer_11/dropout_34/dropout/Mul_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2. ,transformer_encoder/transformer/layer_11/add atransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2c atransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean/reduction_indices˜ Otransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/meanMean0transformer_encoder/transformer/layer_11/add:z:0jtransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2Q Otransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/meanŪ Wtransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/StopGradient StopGradientXtransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Y Wtransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/StopGradient„ \transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/SquaredDifferenceSquaredDifference0transformer_encoder/transformer/layer_11/add:z:0`transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2^ \transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/SquaredDifference˜ etransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2g etransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance/reduction_indicesŌ Stransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/varianceMean`transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/SquaredDifference:z:0ntransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2U Stransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/varianceķ Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2T Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add/y§ Ptransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/addAddV2\transformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/variance:output:0[transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2R Ptransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/addĘ Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/RsqrtRsqrtTtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2T Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/Rsqrtč _transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOphtransformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02a _transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOp¬ Ptransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mulMulVtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/Rsqrt:y:0gtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2R Ptransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul÷ Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Mul0transformer_encoder/transformer/layer_11/add:z:0Ttransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2T Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1Ÿ Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2MulXtransformer_encoder/transformer/layer_11/self_attention_layer_norm/moments/mean:output:0Ttransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2T Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2Ü [transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpReadVariableOpdtransformer_encoder_transformer_layer_11_self_attention_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02] [transformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOpØ Ptransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/subSubctransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/ReadVariableOp:value:0Vtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2R Ptransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/subŸ Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1AddV2Vtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/mul_1:z:0Ttransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2T Rtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1Ę Rtransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOpReadVariableOp[transformer_encoder_transformer_layer_11_intermediate_einsum_einsum_readvariableop_resource* _output_shapes : €€* dtype02T Rtransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOp¬ Ctransformer_encoder/transformer/layer_11/intermediate/einsum/EinsumEinsumVtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0Ztransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum/ReadVariableOp:value:0* N* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* equation  abc,cd->abd2E Ctransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum£ Htransformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOpReadVariableOpQtransformer_encoder_transformer_layer_11_intermediate_add_readvariableop_resource* _output_shapes :€* dtype02J Htransformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOpß 9transformer_encoder/transformer/layer_11/intermediate/addAddV2Ltransformer_encoder/transformer/layer_11/intermediate/einsum/Einsum:output:0Ptransformer_encoder/transformer/layer_11/intermediate/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2; 9transformer_encoder/transformer/layer_11/intermediate/addĮ transformer_encoder/transformer/layer_11/activation_11/Pow:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_11/activation_11/mulĄ :transformer_encoder/transformer/layer_11/activation_11/addAddV2=transformer_encoder/transformer/layer_11/intermediate/add:z:0>transformer_encoder/transformer/layer_11/activation_11/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2< :transformer_encoder/transformer/layer_11/activation_11/addÅ >transformer_encoder/transformer/layer_11/activation_11/mul_1/xConst* _output_shapes :* dtype0* value B **BL?2@ >transformer_encoder/transformer/layer_11/activation_11/mul_1/xĢ transformer_encoder/transformer/layer_11/activation_11/add:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2> transformer_encoder/transformer/layer_11/activation_11/add_1/xConst* _output_shapes :* dtype0* value B *€?2@ >transformer_encoder/transformer/layer_11/activation_11/add_1/xĻ transformer_encoder/transformer/layer_11/activation_11/mul_2/xConst* _output_shapes :* dtype0* value B *?2@ >transformer_encoder/transformer/layer_11/activation_11/mul_2/xĪ abd2? =transformer_encoder/transformer/layer_11/output/einsum/Einsum‘ Btransformer_encoder/transformer/layer_11/output/add/ReadVariableOpReadVariableOpKtransformer_encoder_transformer_layer_11_output_add_readvariableop_resource* _output_shapes :€* dtype02D Btransformer_encoder/transformer/layer_11/output/add/ReadVariableOpĒ 3transformer_encoder/transformer/layer_11/output/addAddV2Ftransformer_encoder/transformer/layer_11/output/einsum/Einsum:output:0Jtransformer_encoder/transformer/layer_11/output/add/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€25 3transformer_encoder/transformer/layer_11/output/addĖ Atransformer_encoder/transformer/layer_11/dropout_35/dropout/ConstConst* _output_shapes :* dtype0* value B *ä8Ž?2C Atransformer_encoder/transformer/layer_11/dropout_35/dropout/ConstĪ ?transformer_encoder/transformer/layer_11/dropout_35/dropout/MulMul7transformer_encoder/transformer/layer_11/output/add:z:0Jtransformer_encoder/transformer/layer_11/dropout_35/dropout/Const:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2A ?transformer_encoder/transformer/layer_11/dropout_35/dropout/Mulķ Atransformer_encoder/transformer/layer_11/dropout_35/dropout/ShapeShape7transformer_encoder/transformer/layer_11/output/add:z:0* T0* _output_shapes :2C Atransformer_encoder/transformer/layer_11/dropout_35/dropout/ShapeŽ Xtransformer_encoder/transformer/layer_11/dropout_35/dropout/random_uniform/RandomUniform RandomUniformJtransformer_encoder/transformer/layer_11/dropout_35/dropout/Shape:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* dtype02Z Xtransformer_encoder/transformer/layer_11/dropout_35/dropout/random_uniform/RandomUniformŻ Jtransformer_encoder/transformer/layer_11/dropout_35/dropout/GreaterEqual/yConst* _output_shapes :* dtype0* value B *ĶĢĢ=2L Jtransformer_encoder/transformer/layer_11/dropout_35/dropout/GreaterEqual/yœ Htransformer_encoder/transformer/layer_11/dropout_35/dropout/GreaterEqual GreaterEqualatransformer_encoder/transformer/layer_11/dropout_35/dropout/random_uniform/RandomUniform:output:0Stransformer_encoder/transformer/layer_11/dropout_35/dropout/GreaterEqual/y:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_11/dropout_35/dropout/GreaterEqual© @transformer_encoder/transformer/layer_11/dropout_35/dropout/CastCastLtransformer_encoder/transformer/layer_11/dropout_35/dropout/GreaterEqual:z:0* DstT0* SrcT0 *5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2B @transformer_encoder/transformer/layer_11/dropout_35/dropout/CastŲ Atransformer_encoder/transformer/layer_11/dropout_35/dropout/Mul_1MulCtransformer_encoder/transformer/layer_11/dropout_35/dropout/Mul:z:0Dtransformer_encoder/transformer/layer_11/dropout_35/dropout/Cast:y:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2C Atransformer_encoder/transformer/layer_11/dropout_35/dropout/Mul_1Č .transformer_encoder/transformer/layer_11/add_1AddV2Etransformer_encoder/transformer/layer_11/dropout_35/dropout/Mul_1:z:0Vtransformer_encoder/transformer/layer_11/self_attention_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€20 .transformer_encoder/transformer/layer_11/add_1€ Ytransformer_encoder/transformer/layer_11/output_layer_norm/moments/mean/reduction_indicesConst* _output_shapes :* dtype0* value B :2[ Ytransformer_encoder/transformer/layer_11/output_layer_norm/moments/mean/reduction_indices‚ Gtransformer_encoder/transformer/layer_11/output_layer_norm/moments/meanMean2transformer_encoder/transformer/layer_11/add_1:z:0btransformer_encoder/transformer/layer_11/output_layer_norm/moments/mean/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2I Gtransformer_encoder/transformer/layer_11/output_layer_norm/moments/meanĆ Otransformer_encoder/transformer/layer_11/output_layer_norm/moments/StopGradient StopGradientPtransformer_encoder/transformer/layer_11/output_layer_norm/moments/mean:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2Q Otransformer_encoder/transformer/layer_11/output_layer_norm/moments/StopGradient Ttransformer_encoder/transformer/layer_11/output_layer_norm/moments/SquaredDifferenceSquaredDifference2transformer_encoder/transformer/layer_11/add_1:z:0Xtransformer_encoder/transformer/layer_11/output_layer_norm/moments/StopGradient:output:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2V Ttransformer_encoder/transformer/layer_11/output_layer_norm/moments/SquaredDifferenceˆ ]transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance/reduction_indicesConst* _output_shapes :* dtype0* value B :2_ ]transformer_encoder/transformer/layer_11/output_layer_norm/moments/variance/reduction_indices“ Ktransformer_encoder/transformer/layer_11/output_layer_norm/moments/varianceMeanXtransformer_encoder/transformer/layer_11/output_layer_norm/moments/SquaredDifference:z:0ftransformer_encoder/transformer/layer_11/output_layer_norm/moments/variance/reduction_indices:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’* keep_dims(2M Ktransformer_encoder/transformer/layer_11/output_layer_norm/moments/varianceŻ Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add/yConst* _output_shapes :* dtype0* value B *̼Œ+2L Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add/y‡ Htransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/addAddV2Ttransformer_encoder/transformer/layer_11/output_layer_norm/moments/variance:output:0Stransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add/y:output:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2J Htransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add® Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/RsqrtRsqrtLtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add:z:0* T0*4 _output_shapes" : ’’’’’’’’’ ’’’’’’’’’2L Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/RsqrtŠ Wtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul/ReadVariableOpReadVariableOp`transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_mul_readvariableop_resource* _output_shapes :€* dtype02Y Wtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul/ReadVariableOpŒ Htransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mulMulNtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/Rsqrt:y:0_transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul/ReadVariableOp:value:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mulį Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_1Mul2transformer_encoder/transformer/layer_11/add_1:z:0Ltransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2L Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_1’ Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_2MulPtransformer_encoder/transformer/layer_11/output_layer_norm/moments/mean:output:0Ltransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2L Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_2Ä Stransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/ReadVariableOpReadVariableOp\transformer_encoder_transformer_layer_11_output_layer_norm_batchnorm_readvariableop_resource* _output_shapes :€* dtype02U Stransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/ReadVariableOpˆ Htransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/subSub[transformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/ReadVariableOp:value:0Ntransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_2:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2J Htransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/sub’ Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1AddV2Ntransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/mul_1:z:0Ltransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/sub:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2L Jtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1µ .transformer_encoder/lambda/strided_slice/stackConst* _output_shapes :* dtype0*! valueB" 20 .transformer_encoder/lambda/strided_slice/stack¹ 0transformer_encoder/lambda/strided_slice/stack_1Const* _output_shapes :* dtype0*! valueB" 22 0transformer_encoder/lambda/strided_slice/stack_1¹ 0transformer_encoder/lambda/strided_slice/stack_2Const* _output_shapes :* dtype0*! valueB" 22 0transformer_encoder/lambda/strided_slice/stack_2Ņ (transformer_encoder/lambda/strided_slice StridedSliceNtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1:z:07transformer_encoder/lambda/strided_slice/stack:output:09transformer_encoder/lambda/strided_slice/stack_1:output:09transformer_encoder/lambda/strided_slice/stack_2:output:0* Index0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€* begin_mask* end_mask2* (transformer_encoder/lambda/strided_sliceŠ "transformer_encoder/lambda/SqueezeSqueeze1transformer_encoder/lambda/strided_slice:output:0* T0*( _output_shapes : ’’’’’’’’’€* squeeze_dims 2$ "transformer_encoder/lambda/Squeezež :transformer_encoder/pooler_transform/MatMul/ReadVariableOpReadVariableOpCtransformer_encoder_pooler_transform_matmul_readvariableop_resource* _output_shapes : €€* dtype02< :transformer_encoder/pooler_transform/MatMul/ReadVariableOpˆ +transformer_encoder/pooler_transform/MatMulMatMul+transformer_encoder/lambda/Squeeze:output:0Btransformer_encoder/pooler_transform/MatMul/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2- +transformer_encoder/pooler_transform/MatMulü ;transformer_encoder/pooler_transform/BiasAdd/ReadVariableOpReadVariableOpDtransformer_encoder_pooler_transform_biasadd_readvariableop_resource* _output_shapes :€* dtype02= ;transformer_encoder/pooler_transform/BiasAdd/ReadVariableOp– ,transformer_encoder/pooler_transform/BiasAddBiasAdd5transformer_encoder/pooler_transform/MatMul:product:0Ctransformer_encoder/pooler_transform/BiasAdd/ReadVariableOp:value:0* T0*( _output_shapes : ’’’’’’’’’€2. ,transformer_encoder/pooler_transform/BiasAddČ )transformer_encoder/pooler_transform/TanhTanh5transformer_encoder/pooler_transform/BiasAdd:output:0* T0*( _output_shapes : ’’’’’’’’’€2+ )transformer_encoder/pooler_transform/Tanh‚ IdentityIdentity-transformer_encoder/pooler_transform/Tanh:y:0* T0*( _output_shapes : ’’’’’’’’’€2 Identity“ Identity_1IdentityNtransformer_encoder/transformer/layer_11/output_layer_norm/batchnorm/add_1:z:0* T0*5 _output_shapes# !: ’’’’’’’’’ ’’’’’’’’’€2 Identity_1" identityIdentity:output:0"! identity_1Identity_1:output:0*… _input_shapesó š: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’: ’’’’’’’’’ ’’’’’’’’’::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/0:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/1:ZV 0 _output_shapes : ’’’’’’’’’ ’’’’’’’’’ " _user_specified_name inputs/2:  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::   _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::  _output_shapes ::   _output_shapes ::!  _output_shapes ::"  _output_shapes ::#  _output_shapes ::$  _output_shapes ::%  _output_shapes ::&  _output_shapes ::'  _output_shapes ::(  _output_shapes ::)  _output_shapes ::*  _output_shapes ::+  _output_shapes ::,  _output_shapes ::-  _output_shapes ::.  _output_shapes ::/  _output_shapes ::0  _output_shapes ::1  _output_shapes ::2  _output_shapes ::3  _output_shapes ::4  _output_shapes ::5  _output_shapes ::6  _output_shapes ::7  _output_shapes ::8  _output_shapes ::9  _output_shapes :::  _output_shapes ::;  _output_shapes ::<  _output_shapes ::=  _output_shapes ::>  _output_shapes ::?  _output_shapes ::@  _output_shapes ::A  _output_shapes ::B  _output_shapes ::C  _output_shapes ::D  _output_shapes ::E  _output_shapes ::F  _output_shapes ::G  _output_shapes ::H  _output_shapes ::I  _output_shapes ::J  _output_shapes ::K  _output_shapes ::L  _output_shapes ::M  _output_shapes ::N  _output_shapes ::O  _output_shapes ::P  _output_shapes ::Q  _output_shapes ::R  _output_shapes ::S  _output_shapes ::T  _output_shapes ::U  _output_shapes ::V  _output_shapes ::W  _output_shapes ::X  _output_shapes ::Y  _output_shapes ::Z  _output_shapes ::[  _output_shapes ::\  _output_shapes ::]  _output_shapes ::^  _output_shapes ::_  _output_shapes ::`  _output_shapes ::a  _output_shapes ::b  _output_shapes ::c  _output_shapes ::d  _output_shapes ::e  _output_shapes ::f  _output_shapes ::g  _output_shapes ::h  _output_shapes ::i  _output_shapes ::j  _output_shapes ::k  _output_shapes ::l  _output_shapes ::m  _output_shapes ::n  _output_shapes ::o  _output_shapes ::p  _output_shapes ::q  _output_shapes ::r  _output_shapes ::s  _output_shapes ::t  _output_shapes ::u  _output_shapes ::v  _output_shapes ::w  _output_shapes ::x  _output_shapes ::y  _output_shapes ::z  _output_shapes ::{  _output_shapes ::|  _output_shapes ::}  _output_shapes ::~  _output_shapes ::  _output_shapes ::€  _output_shapes ::  _output_shapes ::‚  _output_shapes ::ƒ  _output_shapes ::„  _output_shapes ::…  _output_shapes ::†  _output_shapes ::‡  _output_shapes ::ˆ  _output_shapes ::‰  _output_shapes ::Š  _output_shapes ::‹  _output_shapes ::Œ  _output_shapes ::  _output_shapes ::Ž  _output_shapes ::  _output_shapes ::  _output_shapes ::‘  _output_shapes ::’  _output_shapes ::“  _output_shapes ::”  _output_shapes ::•  _output_shapes ::–  _output_shapes ::—  _output_shapes ::˜  _output_shapes ::™  _output_shapes ::š  _output_shapes ::›  _output_shapes ::œ  _output_shapes ::  _output_shapes ::ž  _output_shapes ::Ÿ  _output_shapes ::   _output_shapes ::”  _output_shapes ::¢  _output_shapes ::£  _output_shapes ::¤  _output_shapes ::„  _output_shapes ::¦  _output_shapes ::§  _output_shapes ::Ø  _output_shapes ::©  _output_shapes ::Ŗ  _output_shapes ::«  _output_shapes ::¬  _output_shapes ::­  _output_shapes ::®  _output_shapes ::Æ  _output_shapes ::°  _output_shapes ::±  _output_shapes ::²  _output_shapes ::³  _output_shapes ::“  _output_shapes ::µ  _output_shapes ::¶  _output_shapes ::·  _output_shapes ::ø  _output_shapes ::¹  _output_shapes ::ŗ  _output_shapes ::»  _output_shapes ::¼  _output_shapes ::½  _output_shapes ::¾  _output_shapes ::æ  _output_shapes ::Ą  _output_shapes ::Į  _output_shapes ::Ā  _output_shapes ::Ć  _output_shapes ::Ä  _output_shapes ::Å  _output_shapes ::Ę  _output_shapes ::Ē  _output_shapes ::Č  _output_shapes ::É  _output_shapes :"­ L saver_filename:0StatefulPartitionedCall_1:0StatefulPartitionedCall_28" saved_model_main_op  NoOp*> __saved_model_init_op%# __saved_model_init_op NoOp*Ē serving_default³ J input_mask< serving_default_input_mask:0 ’’’’’’’’’ ’’’’’’’’’ R input_type_ids@ serving_default_input_type_ids:0 ’’’’’’’’’ ’’’’’’’’’ R input_word_ids@ serving_default_input_word_ids:0 ’’’’’’’’’ ’’’’’’’’’H transformer_encoder1 StatefulPartitionedCall:0 ’’’’’’’’’€W transformer_encoder_1> StatefulPartitionedCall:1 ’’’’’’’’’ ’’’’’’’’’€tensorflow/serving/predict2-  asset_path_initializer:0cased_vocab.txt:’Ø õ layer-0 layer-1 layer-2 layer_with_weights-0 layer-3  vocab_file  do_lower_case  variables regularization_losses  trainable_variables   keras_api   signatures +Ė&call_and_return_all_conditional_losses Ģ__call__ Ķ_default_save_signature"Å _tf_keras_network©{"class_name": "Functional", "name": "functional_1", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "config": {"name": "functional_1", "layers": [{"class_name": "InputLayer", "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_word_ids"}, "name": "input_word_ids", "inbound_nodes": []}, {"class_name": "InputLayer", "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_mask"}, "name": "input_mask", "inbound_nodes": []}, {"class_name": "InputLayer", "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_type_ids"}, "name": "input_type_ids", "inbound_nodes": []}, {"class_name": "Text>TransformerEncoder", "config": {"vocab_size": 501153, "hidden_size": 768, "num_layers": 12, "num_attention_heads": 12, "sequence_length": null, "max_sequence_length": 512, "type_vocab_size": 2, "intermediate_size": 3072, "activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "return_all_encoder_outputs": false, "output_range": null, "embedding_width": null}, "name": "transformer_encoder", "inbound_nodes": [[["input_word_ids", 0, 0, {}], ["input_mask", 0, 0, {}], ["input_type_ids", 0, 0, {}]]]}], "input_layers": [["input_word_ids", 0, 0], ["input_mask", 0, 0], ["input_type_ids", 0, 0]], "output_layers": [["transformer_encoder", 1, 1], ["transformer_encoder", 1, 0]]}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null]}, {"class_name": "TensorShape", "items": [null, null]}, {"class_name": "TensorShape", "items": [null, null]}], "is_graph_network": true, "keras_version": "2.3.0-tf", "backend": "tensorflow", "model_config": {"class_name": "Functional", "config": {"name": "functional_1", "layers": [{"class_name": "InputLayer", "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_word_ids"}, "name": "input_word_ids", "inbound_nodes": []}, {"class_name": "InputLayer", "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_mask"}, "name": "input_mask", "inbound_nodes": []}, {"class_name": "InputLayer", "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_type_ids"}, "name": "input_type_ids", "inbound_nodes": []}, {"class_name": "Text>TransformerEncoder", "config": {"vocab_size": 501153, "hidden_size": 768, "num_layers": 12, "num_attention_heads": 12, "sequence_length": null, "max_sequence_length": 512, "type_vocab_size": 2, "intermediate_size": 3072, "activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "return_all_encoder_outputs": false, "output_range": null, "embedding_width": null}, "name": "transformer_encoder", "inbound_nodes": [[["input_word_ids", 0, 0, {}], ["input_mask", 0, 0, {}], ["input_type_ids", 0, 0, {}]]]}], "input_layers": [["input_word_ids", 0, 0], ["input_mask", 0, 0], ["input_type_ids", 0, 0]], "output_layers": [["transformer_encoder", 1, 1], ["transformer_encoder", 1, 0]]}}} ł"ö _tf_keras_input_layerÖ{"class_name": "InputLayer", "name": "input_word_ids", "dtype": "int32", "sparse": false, "ragged": false, "batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_word_ids"}} ń"ī _tf_keras_input_layerĪ{"class_name": "InputLayer", "name": "input_mask", "dtype": "int32", "sparse": false, "ragged": false, "batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_mask"}} ł"ö _tf_keras_input_layerÖ{"class_name": "InputLayer", "name": "input_type_ids", "dtype": "int32", "sparse": false, "ragged": false, "batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_type_ids"}} Ą  layer-0  layer_with_weights-0  layer-1 layer-2 layer_with_weights-1 layer-3 layer_with_weights-2 layer-4 layer-5 layer_with_weights-3 layer-6 layer-7 layer-8 layer-9 layer_with_weights-4 layer-10 layer_with_weights-5 layer-11 layer_with_weights-6 layer-12 layer_with_weights-7 layer-13 layer_with_weights-8 layer-14 layer_with_weights-9 layer-15 layer_with_weights-10 layer-16 layer_with_weights-11 layer-17 layer_with_weights-12 layer-18 layer_with_weights-13 layer-19  layer_with_weights-14  layer-20 !layer_with_weights-15 !layer-21 "layer-22 #layer_with_weights-16 #layer-23 $ variables %regularization_losses &trainable_variables ' keras_api +Ī&call_and_return_all_conditional_losses Ļ__call__"Ø _tf_keras_networkŒ {"class_name": "TransformerEncoder", "name": "transformer_encoder", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "config": {"vocab_size": 501153, "hidden_size": 768, "num_layers": 12, "num_attention_heads": 12, "sequence_length": null, "max_sequence_length": 512, "type_vocab_size": 2, "intermediate_size": 3072, "activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "return_all_encoder_outputs": false, "output_range": null, "embedding_width": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null]}, {"class_name": "TensorShape", "items": [null, null]}, {"class_name": "TensorShape", "items": [null, null]}], "is_graph_network": true, "keras_version": "2.3.0-tf", "backend": "tensorflow", "model_config": {"class_name": "TransformerEncoder", "config": {"vocab_size": 501153, "hidden_size": 768, "num_layers": 12, "num_attention_heads": 12, "sequence_length": null, "max_sequence_length": 512, "type_vocab_size": 2, "intermediate_size": 3072, "activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "return_all_encoder_outputs": false, "output_range": null, "embedding_width": null}}} * : 2Variable © (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198 199" trackable_list_wrapper " trackable_list_wrapper   (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198" trackable_list_wrapper Ó ļ layer_metrics  variables šnon_trainable_variables ńlayers regularization_losses ņmetrics ólayer_regularization_losses  trainable_variables Ģ__call__ Ķ_default_save_signature +Ė&call_and_return_all_conditional_losses 'Ė"call_and_return_conditional_losses" _generic_user_object - Šserving_default" signature_map ł"ö _tf_keras_input_layerÖ{"class_name": "InputLayer", "name": "input_word_ids", "dtype": "int32", "sparse": false, "ragged": false, "batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_word_ids"}} Ś ( embeddings ō variables õregularization_losses ötrainable_variables ÷ keras_api +Ń&call_and_return_all_conditional_losses Ņ__call__"µ _tf_keras_layer›{"class_name": "OnDeviceEmbedding", "name": "word_embeddings", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "word_embeddings", "trainable": true, "dtype": "float32", "vocab_size": 501153, "embedding_width": 768, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}, "__passive_serialization__": true}, "use_one_hot": false}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null]}} ł"ö _tf_keras_input_layerÖ{"class_name": "InputLayer", "name": "input_type_ids", "dtype": "int32", "sparse": false, "ragged": false, "batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_type_ids"}} Ń ) embeddings )_position_embeddings ų variables łregularization_losses śtrainable_variables ū keras_api +Ó&call_and_return_all_conditional_losses Ō__call__"’ _tf_keras_layerų{"class_name": "PositionEmbedding", "name": "position_embedding", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "position_embedding", "trainable": true, "dtype": "float32", "max_sequence_length": 512, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "use_dynamic_slicing": true}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ō * embeddings ü variables żregularization_losses žtrainable_variables ’ keras_api +Õ&call_and_return_all_conditional_losses Ö__call__"Æ _tf_keras_layer•{"class_name": "OnDeviceEmbedding", "name": "type_embeddings", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "type_embeddings", "trainable": true, "dtype": "float32", "vocab_size": 2, "embedding_width": 768, "initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}, "__passive_serialization__": true}, "use_one_hot": true}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null]}} Ń € variables regularization_losses ‚trainable_variables ƒ keras_api +×&call_and_return_all_conditional_losses Ų__call__"¼ _tf_keras_layer¢{"class_name": "Add", "name": "add", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "add", "trainable": true, "dtype": "float32"}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} Č „axis +gamma ,beta … variables †regularization_losses ‡trainable_variables ˆ keras_api +Ł&call_and_return_all_conditional_losses Ś__call__"“ _tf_keras_layerł{"class_name": "LayerNormalization", "name": "embeddings/layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "embeddings/layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ä ‰ variables Šregularization_losses ‹trainable_variables Œ keras_api +Ū&call_and_return_all_conditional_losses Ü__call__"Æ _tf_keras_layer•{"class_name": "Dropout", "name": "dropout", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ń"ī _tf_keras_input_layerĪ{"class_name": "InputLayer", "name": "input_mask", "dtype": "int32", "sparse": false, "ragged": false, "batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "config": {"batch_input_shape": {"class_name": "__tuple__", "items": [null, null]}, "dtype": "int32", "sparse": false, "ragged": false, "name": "input_mask"}} ·  variables Žregularization_losses trainable_variables  keras_api +Ż&call_and_return_all_conditional_losses Ž__call__"¢ _tf_keras_layerˆ{"class_name": "SelfAttentionMask", "name": "self_attention_mask", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_mask", "trainable": true, "dtype": "float32"}} † ‘_attention_layer ’_attention_output_dense “_attention_dropout ”_attention_layer_norm •_intermediate_dense #–_intermediate_activation_layer — _output_dense ˜_output_dropout ™_output_layer_norm š variables ›regularization_losses œtrainable_variables  keras_api +ß&call_and_return_all_conditional_losses ą__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_0", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_0", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † ž_attention_layer Ÿ_attention_output_dense  _attention_dropout ”_attention_layer_norm ¢_intermediate_dense #£_intermediate_activation_layer ¤ _output_dense „_output_dropout ¦_output_layer_norm § variables Øregularization_losses ©trainable_variables Ŗ keras_api +į&call_and_return_all_conditional_losses ā__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_1", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_1", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † «_attention_layer ¬_attention_output_dense ­_attention_dropout ®_attention_layer_norm Æ_intermediate_dense #°_intermediate_activation_layer ± _output_dense ²_output_dropout ³_output_layer_norm “ variables µregularization_losses ¶trainable_variables · keras_api +ć&call_and_return_all_conditional_losses ä__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_2", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_2", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † ø_attention_layer ¹_attention_output_dense ŗ_attention_dropout »_attention_layer_norm ¼_intermediate_dense #½_intermediate_activation_layer ¾ _output_dense æ_output_dropout Ą_output_layer_norm Į variables Āregularization_losses Ćtrainable_variables Ä keras_api +å&call_and_return_all_conditional_losses ę__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_3", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_3", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † Å_attention_layer Ę_attention_output_dense Ē_attention_dropout Č_attention_layer_norm É_intermediate_dense #Ź_intermediate_activation_layer Ė _output_dense Ģ_output_dropout Ķ_output_layer_norm Ī variables Ļregularization_losses Štrainable_variables Ń keras_api +ē&call_and_return_all_conditional_losses č__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_4", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_4", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † Ņ_attention_layer Ó_attention_output_dense Ō_attention_dropout Õ_attention_layer_norm Ö_intermediate_dense #×_intermediate_activation_layer Ų _output_dense Ł_output_dropout Ś_output_layer_norm Ū variables Üregularization_losses Żtrainable_variables Ž keras_api +é&call_and_return_all_conditional_losses ź__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_5", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_5", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † ß_attention_layer ą_attention_output_dense į_attention_dropout ā_attention_layer_norm ć_intermediate_dense #ä_intermediate_activation_layer å _output_dense ę_output_dropout ē_output_layer_norm č variables éregularization_losses źtrainable_variables ė keras_api +ė&call_and_return_all_conditional_losses ģ__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_6", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_6", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † ģ_attention_layer ķ_attention_output_dense ī_attention_dropout ļ_attention_layer_norm š_intermediate_dense #ń_intermediate_activation_layer ņ _output_dense ó_output_dropout ō_output_layer_norm õ variables öregularization_losses ÷trainable_variables ų keras_api +ķ&call_and_return_all_conditional_losses ī__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_7", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_7", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † ł_attention_layer ś_attention_output_dense ū_attention_dropout ü_attention_layer_norm ż_intermediate_dense #ž_intermediate_activation_layer ’ _output_dense €_output_dropout _output_layer_norm ‚ variables ƒregularization_losses „trainable_variables … keras_api +ļ&call_and_return_all_conditional_losses š__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_8", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_8", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} † †_attention_layer ‡_attention_output_dense ˆ_attention_dropout ‰_attention_layer_norm Š_intermediate_dense #‹_intermediate_activation_layer Œ _output_dense _output_dropout Ž_output_layer_norm  variables regularization_losses ‘trainable_variables ’ keras_api +ń&call_and_return_all_conditional_losses ņ__call__"… _tf_keras_layerė{"class_name": "Transformer", "name": "transformer/layer_9", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_9", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} ˆ “_attention_layer ”_attention_output_dense •_attention_dropout –_attention_layer_norm —_intermediate_dense #˜_intermediate_activation_layer ™ _output_dense š_output_dropout ›_output_layer_norm œ variables regularization_losses žtrainable_variables Ÿ keras_api +ó&call_and_return_all_conditional_losses ō__call__"‡ _tf_keras_layerķ{"class_name": "Transformer", "name": "transformer/layer_10", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_10", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} ˆ  _attention_layer ”_attention_output_dense ¢_attention_dropout £_attention_layer_norm ¤_intermediate_dense #„_intermediate_activation_layer ¦ _output_dense §_output_dropout Ø_output_layer_norm © variables Ŗregularization_losses «trainable_variables ¬ keras_api +õ&call_and_return_all_conditional_losses ö__call__"‡ _tf_keras_layerķ{"class_name": "Transformer", "name": "transformer/layer_11", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "transformer/layer_11", "trainable": true, "dtype": "float32", "num_attention_heads": 12, "intermediate_size": 3072, "intermediate_activation": "Text>gelu", "dropout_rate": 0.1, "attention_dropout_rate": 0.1, "output_range": null, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, null]}]} ˜ ­ variables ®regularization_losses Ætrainable_variables ° keras_api +÷&call_and_return_all_conditional_losses ų__call__"ƒ _tf_keras_layeré{"class_name": "Lambda", "name": "lambda", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "lambda", "trainable": true, "dtype": "float32", "function": {"class_name": "__tuple__", "items": ["4wEAAAAAAAAAAQAAAAYAAABTAAAAcyQAAAB0AGoBfABkAGQAhQJkAWQChQJkAGQAhQJmAxkAZAJk\nA40CUwApBE7pAAAAAOkBAAAAKQHaBGF4aXMpAtoCdGbaB3NxdWVlemUpAdoBeKkAcgcAAAD62i9n\nb29nbGUvc3JjL2Nsb3VkL2Zhbmd4aWFveXUvdHJhbnNsYXRlL2dvb2dsZTMvYmxhemUtYmluL3Ro\naXJkX3BhcnR5L3RlbnNvcmZsb3dfbW9kZWxzL29mZmljaWFsL25scC9iZXJ0L2V4cG9ydF90Zmh1\nYi5ydW5maWxlcy9nb29nbGUzL3RoaXJkX3BhcnR5L3RlbnNvcmZsb3dfbW9kZWxzL29mZmljaWFs\nL25scC9tb2RlbGluZy9uZXR3b3Jrcy90cmFuc2Zvcm1lcl9lbmNvZGVyLnB52gg8bGFtYmRhPsEA\nAABzAAAAAA==\n", null, null]}, "function_type": "lambda", "module": "google3.third_party.tensorflow_models.official.nlp.modeling.networks.transformer_encoder", "output_shape": null, "output_shape_type": "raw", "output_shape_module": null, "arguments": {}}} ‰ ķkernel ībias ± variables ²regularization_losses ³trainable_variables “ keras_api +ł&call_and_return_all_conditional_losses ś__call__"Ü _tf_keras_layerĀ{"class_name": "Dense", "name": "pooler_transform", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "pooler_transform", "trainable": true, "dtype": "float32", "units": 768, "activation": "tanh", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "input_spec": {"class_name": "InputSpec", "config": {"dtype": null, "shape": null, "ndim": null, "max_ndim": null, "min_ndim": 2, "axes": {"-1": 768}}}, "build_input_shape": {"class_name": "TensorShape", "items": [null, 768]}}   (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198" trackable_list_wrapper " trackable_list_wrapper   (0 )1 *2 +3 ,4 -5 .6 /7 08 19 210 311 412 513 614 715 816 917 :18 ;19 <20 =21 >22 ?23 @24 A25 B26 C27 D28 E29 F30 G31 H32 I33 J34 K35 L36 M37 N38 O39 P40 Q41 R42 S43 T44 U45 V46 W47 X48 Y49 Z50 [51 \52 ]53 ^54 _55 `56 a57 b58 c59 d60 e61 f62 g63 h64 i65 j66 k67 l68 m69 n70 o71 p72 q73 r74 s75 t76 u77 v78 w79 x80 y81 z82 {83 |84 }85 ~86 87 €88 89 ‚90 ƒ91 „92 …93 †94 ‡95 ˆ96 ‰97 Š98 ‹99 Œ100 101 Ž102 103 104 ‘105 ’106 “107 ”108 •109 –110 —111 ˜112 ™113 š114 ›115 œ116 117 ž118 Ÿ119  120 ”121 ¢122 £123 ¤124 „125 ¦126 §127 Ø128 ©129 Ŗ130 «131 ¬132 ­133 ®134 Æ135 °136 ±137 ²138 ³139 “140 µ141 ¶142 ·143 ø144 ¹145 ŗ146 »147 ¼148 ½149 ¾150 æ151 Ą152 Į153 Ā154 Ć155 Ä156 Å157 Ę158 Ē159 Č160 É161 Ź162 Ė163 Ģ164 Ķ165 Ī166 Ļ167 Š168 Ń169 Ņ170 Ó171 Ō172 Õ173 Ö174 ×175 Ų176 Ł177 Ś178 Ū179 Ü180 Ż181 Ž182 ß183 ą184 į185 ā186 ć187 ä188 å189 ę190 ē191 č192 é193 ź194 ė195 ģ196 ķ197 ī198" trackable_list_wrapper µ µ layer_metrics $ variables ¶non_trainable_variables ·layers %regularization_losses ømetrics ¹layer_regularization_losses &trainable_variables Ļ__call__ +Ī&call_and_return_all_conditional_losses 'Ī"call_and_return_conditional_losses" _generic_user_object /:- ”Ė€2word_embeddings/embeddings 1:/ €€2position_embedding/embeddings -:+ €2type_embeddings/embeddings *:(€2embeddings/layer_norm/gamma ):'€2embeddings/layer_norm/beta F:D € @2/transformer/layer_0/self_attention/query/kernel ?:= @2-transformer/layer_0/self_attention/query/bias D:B € @2-transformer/layer_0/self_attention/key/kernel =:; @2+transformer/layer_0/self_attention/key/bias F:D € @2/transformer/layer_0/self_attention/value/kernel ?:= @2-transformer/layer_0/self_attention/value/bias Q:O  @€2:transformer/layer_0/self_attention/attention_output/kernel G:E€28transformer/layer_0/self_attention/attention_output/bias B:@€23transformer/layer_0/self_attention_layer_norm/gamma A:?€22transformer/layer_0/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_0/intermediate/kernel 4:2€2%transformer/layer_0/intermediate/bias 5:3 €€2!transformer/layer_0/output/kernel .:,€2transformer/layer_0/output/bias ::8€2+transformer/layer_0/output_layer_norm/gamma 9:7€2*transformer/layer_0/output_layer_norm/beta F:D € @2/transformer/layer_1/self_attention/query/kernel ?:= @2-transformer/layer_1/self_attention/query/bias D:B € @2-transformer/layer_1/self_attention/key/kernel =:; @2+transformer/layer_1/self_attention/key/bias F:D € @2/transformer/layer_1/self_attention/value/kernel ?:= @2-transformer/layer_1/self_attention/value/bias Q:O  @€2:transformer/layer_1/self_attention/attention_output/kernel G:E€28transformer/layer_1/self_attention/attention_output/bias B:@€23transformer/layer_1/self_attention_layer_norm/gamma A:?€22transformer/layer_1/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_1/intermediate/kernel 4:2€2%transformer/layer_1/intermediate/bias 5:3 €€2!transformer/layer_1/output/kernel .:,€2transformer/layer_1/output/bias ::8€2+transformer/layer_1/output_layer_norm/gamma 9:7€2*transformer/layer_1/output_layer_norm/beta F:D € @2/transformer/layer_2/self_attention/query/kernel ?:= @2-transformer/layer_2/self_attention/query/bias D:B € @2-transformer/layer_2/self_attention/key/kernel =:; @2+transformer/layer_2/self_attention/key/bias F:D € @2/transformer/layer_2/self_attention/value/kernel ?:= @2-transformer/layer_2/self_attention/value/bias Q:O  @€2:transformer/layer_2/self_attention/attention_output/kernel G:E€28transformer/layer_2/self_attention/attention_output/bias B:@€23transformer/layer_2/self_attention_layer_norm/gamma A:?€22transformer/layer_2/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_2/intermediate/kernel 4:2€2%transformer/layer_2/intermediate/bias 5:3 €€2!transformer/layer_2/output/kernel .:,€2transformer/layer_2/output/bias ::8€2+transformer/layer_2/output_layer_norm/gamma 9:7€2*transformer/layer_2/output_layer_norm/beta F:D € @2/transformer/layer_3/self_attention/query/kernel ?:= @2-transformer/layer_3/self_attention/query/bias D:B € @2-transformer/layer_3/self_attention/key/kernel =:; @2+transformer/layer_3/self_attention/key/bias F:D € @2/transformer/layer_3/self_attention/value/kernel ?:= @2-transformer/layer_3/self_attention/value/bias Q:O  @€2:transformer/layer_3/self_attention/attention_output/kernel G:E€28transformer/layer_3/self_attention/attention_output/bias B:@€23transformer/layer_3/self_attention_layer_norm/gamma A:?€22transformer/layer_3/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_3/intermediate/kernel 4:2€2%transformer/layer_3/intermediate/bias 5:3 €€2!transformer/layer_3/output/kernel .:,€2transformer/layer_3/output/bias ::8€2+transformer/layer_3/output_layer_norm/gamma 9:7€2*transformer/layer_3/output_layer_norm/beta F:D € @2/transformer/layer_4/self_attention/query/kernel ?:= @2-transformer/layer_4/self_attention/query/bias D:B € @2-transformer/layer_4/self_attention/key/kernel =:; @2+transformer/layer_4/self_attention/key/bias F:D € @2/transformer/layer_4/self_attention/value/kernel ?:= @2-transformer/layer_4/self_attention/value/bias Q:O  @€2:transformer/layer_4/self_attention/attention_output/kernel G:E€28transformer/layer_4/self_attention/attention_output/bias B:@€23transformer/layer_4/self_attention_layer_norm/gamma A:?€22transformer/layer_4/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_4/intermediate/kernel 4:2€2%transformer/layer_4/intermediate/bias 5:3 €€2!transformer/layer_4/output/kernel .:,€2transformer/layer_4/output/bias ::8€2+transformer/layer_4/output_layer_norm/gamma 9:7€2*transformer/layer_4/output_layer_norm/beta F:D € @2/transformer/layer_5/self_attention/query/kernel ?:= @2-transformer/layer_5/self_attention/query/bias D:B € @2-transformer/layer_5/self_attention/key/kernel =:; @2+transformer/layer_5/self_attention/key/bias F:D € @2/transformer/layer_5/self_attention/value/kernel ?:= @2-transformer/layer_5/self_attention/value/bias Q:O  @€2:transformer/layer_5/self_attention/attention_output/kernel G:E€28transformer/layer_5/self_attention/attention_output/bias B:@€23transformer/layer_5/self_attention_layer_norm/gamma A:?€22transformer/layer_5/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_5/intermediate/kernel 4:2€2%transformer/layer_5/intermediate/bias 5:3 €€2!transformer/layer_5/output/kernel .:,€2transformer/layer_5/output/bias ::8€2+transformer/layer_5/output_layer_norm/gamma 9:7€2*transformer/layer_5/output_layer_norm/beta F:D € @2/transformer/layer_6/self_attention/query/kernel ?:= @2-transformer/layer_6/self_attention/query/bias D:B € @2-transformer/layer_6/self_attention/key/kernel =:; @2+transformer/layer_6/self_attention/key/bias F:D € @2/transformer/layer_6/self_attention/value/kernel ?:= @2-transformer/layer_6/self_attention/value/bias Q:O  @€2:transformer/layer_6/self_attention/attention_output/kernel G:E€28transformer/layer_6/self_attention/attention_output/bias B:@€23transformer/layer_6/self_attention_layer_norm/gamma A:?€22transformer/layer_6/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_6/intermediate/kernel 4:2€2%transformer/layer_6/intermediate/bias 5:3 €€2!transformer/layer_6/output/kernel .:,€2transformer/layer_6/output/bias ::8€2+transformer/layer_6/output_layer_norm/gamma 9:7€2*transformer/layer_6/output_layer_norm/beta F:D € @2/transformer/layer_7/self_attention/query/kernel ?:= @2-transformer/layer_7/self_attention/query/bias D:B € @2-transformer/layer_7/self_attention/key/kernel =:; @2+transformer/layer_7/self_attention/key/bias F:D € @2/transformer/layer_7/self_attention/value/kernel ?:= @2-transformer/layer_7/self_attention/value/bias Q:O  @€2:transformer/layer_7/self_attention/attention_output/kernel G:E€28transformer/layer_7/self_attention/attention_output/bias B:@€23transformer/layer_7/self_attention_layer_norm/gamma A:?€22transformer/layer_7/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_7/intermediate/kernel 4:2€2%transformer/layer_7/intermediate/bias 5:3 €€2!transformer/layer_7/output/kernel .:,€2transformer/layer_7/output/bias ::8€2+transformer/layer_7/output_layer_norm/gamma 9:7€2*transformer/layer_7/output_layer_norm/beta F:D € @2/transformer/layer_8/self_attention/query/kernel ?:= @2-transformer/layer_8/self_attention/query/bias D:B € @2-transformer/layer_8/self_attention/key/kernel =:; @2+transformer/layer_8/self_attention/key/bias F:D € @2/transformer/layer_8/self_attention/value/kernel ?:= @2-transformer/layer_8/self_attention/value/bias Q:O  @€2:transformer/layer_8/self_attention/attention_output/kernel G:E€28transformer/layer_8/self_attention/attention_output/bias B:@€23transformer/layer_8/self_attention_layer_norm/gamma A:?€22transformer/layer_8/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_8/intermediate/kernel 4:2€2%transformer/layer_8/intermediate/bias 5:3 €€2!transformer/layer_8/output/kernel .:,€2transformer/layer_8/output/bias ::8€2+transformer/layer_8/output_layer_norm/gamma 9:7€2*transformer/layer_8/output_layer_norm/beta F:D € @2/transformer/layer_9/self_attention/query/kernel ?:= @2-transformer/layer_9/self_attention/query/bias D:B € @2-transformer/layer_9/self_attention/key/kernel =:; @2+transformer/layer_9/self_attention/key/bias F:D € @2/transformer/layer_9/self_attention/value/kernel ?:= @2-transformer/layer_9/self_attention/value/bias Q:O  @€2:transformer/layer_9/self_attention/attention_output/kernel G:E€28transformer/layer_9/self_attention/attention_output/bias B:@€23transformer/layer_9/self_attention_layer_norm/gamma A:?€22transformer/layer_9/self_attention_layer_norm/beta ;:9 €€2'transformer/layer_9/intermediate/kernel 4:2€2%transformer/layer_9/intermediate/bias 5:3 €€2!transformer/layer_9/output/kernel .:,€2transformer/layer_9/output/bias ::8€2+transformer/layer_9/output_layer_norm/gamma 9:7€2*transformer/layer_9/output_layer_norm/beta G:E € @20transformer/layer_10/self_attention/query/kernel @:> @2.transformer/layer_10/self_attention/query/bias E:C € @2.transformer/layer_10/self_attention/key/kernel >:< @2,transformer/layer_10/self_attention/key/bias G:E € @20transformer/layer_10/self_attention/value/kernel @:> @2.transformer/layer_10/self_attention/value/bias R:P  @€2;transformer/layer_10/self_attention/attention_output/kernel H:F€29transformer/layer_10/self_attention/attention_output/bias C:A€24transformer/layer_10/self_attention_layer_norm/gamma B:@€23transformer/layer_10/self_attention_layer_norm/beta <:: €€2(transformer/layer_10/intermediate/kernel 5:3€2&transformer/layer_10/intermediate/bias 6:4 €€2"transformer/layer_10/output/kernel /:-€2 transformer/layer_10/output/bias ;:9€2,transformer/layer_10/output_layer_norm/gamma ::8€2+transformer/layer_10/output_layer_norm/beta G:E € @20transformer/layer_11/self_attention/query/kernel @:> @2.transformer/layer_11/self_attention/query/bias E:C € @2.transformer/layer_11/self_attention/key/kernel >:< @2,transformer/layer_11/self_attention/key/bias G:E € @20transformer/layer_11/self_attention/value/kernel @:> @2.transformer/layer_11/self_attention/value/bias R:P  @€2;transformer/layer_11/self_attention/attention_output/kernel H:F€29transformer/layer_11/self_attention/attention_output/bias C:A€24transformer/layer_11/self_attention_layer_norm/gamma B:@€23transformer/layer_11/self_attention_layer_norm/beta <:: €€2(transformer/layer_11/intermediate/kernel 5:3€2&transformer/layer_11/intermediate/bias 6:4 €€2"transformer/layer_11/output/kernel /:-€2 transformer/layer_11/output/bias ;:9€2,transformer/layer_11/output_layer_norm/gamma ::8€2+transformer/layer_11/output_layer_norm/beta +:) €€2pooler_transform/kernel $:"€2pooler_transform/bias " trackable_dict_wrapper ' 0" trackable_list_wrapper < 0 1 2 3" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ' (0" trackable_list_wrapper " trackable_list_wrapper ' (0" trackable_list_wrapper ø ŗ layer_metrics ō variables »non_trainable_variables ¼layers õregularization_losses ½metrics ¾layer_regularization_losses ötrainable_variables Ņ__call__ +Ń&call_and_return_all_conditional_losses 'Ń"call_and_return_conditional_losses" _generic_user_object ' )0" trackable_list_wrapper " trackable_list_wrapper ' )0" trackable_list_wrapper ø æ layer_metrics ų variables Ąnon_trainable_variables Įlayers łregularization_losses Āmetrics Ćlayer_regularization_losses śtrainable_variables Ō__call__ +Ó&call_and_return_all_conditional_losses 'Ó"call_and_return_conditional_losses" _generic_user_object ' *0" trackable_list_wrapper " trackable_list_wrapper ' *0" trackable_list_wrapper ø Ä layer_metrics ü variables Ånon_trainable_variables Ęlayers żregularization_losses Ēmetrics Člayer_regularization_losses žtrainable_variables Ö__call__ +Õ&call_and_return_all_conditional_losses 'Õ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø É layer_metrics € variables Źnon_trainable_variables Ėlayers regularization_losses Ģmetrics Ķlayer_regularization_losses ‚trainable_variables Ų__call__ +×&call_and_return_all_conditional_losses '×"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . +0 ,1" trackable_list_wrapper " trackable_list_wrapper . +0 ,1" trackable_list_wrapper ø Ī layer_metrics … variables Ļnon_trainable_variables Šlayers †regularization_losses Ńmetrics Ņlayer_regularization_losses ‡trainable_variables Ś__call__ +Ł&call_and_return_all_conditional_losses 'Ł"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ó layer_metrics ‰ variables Ōnon_trainable_variables Õlayers Šregularization_losses Ömetrics ×layer_regularization_losses ‹trainable_variables Ü__call__ +Ū&call_and_return_all_conditional_losses 'Ū"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ų layer_metrics  variables Łnon_trainable_variables Ślayers Žregularization_losses Ūmetrics Ülayer_regularization_losses trainable_variables Ž__call__ +Ż&call_and_return_all_conditional_losses 'Ż"call_and_return_conditional_losses" _generic_user_object é Ż _query_dense Ž _key_dense ß _value_dense ą_masked_softmax į_dropout_layer ’ _output_dense ā variables ćregularization_losses ätrainable_variables å keras_api +ū&call_and_return_all_conditional_losses ü__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} Ž ępartial_output_shape ēfull_output_shape 3kernel 4bias č variables éregularization_losses źtrainable_variables ė keras_api +ż&call_and_return_all_conditional_losses ž__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Č ģ variables ķregularization_losses ītrainable_variables ļ keras_api +’&call_and_return_all_conditional_losses €__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_1", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_1", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Š šaxis 5gamma 6beta ń variables ņregularization_losses ótrainable_variables ō keras_api +&call_and_return_all_conditional_losses ‚__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ų 7kernel 7_kernel 8bias 8_bias õ variables öregularization_losses ÷trainable_variables ų keras_api +ƒ&call_and_return_all_conditional_losses „__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¹ ł variables śregularization_losses ūtrainable_variables ü keras_api +…&call_and_return_all_conditional_losses †__call__"¤ _tf_keras_layerŠ{"class_name": "Activation", "name": "activation", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Ģ 9kernel 9_kernel :bias :_bias ż variables žregularization_losses ’trainable_variables € keras_api +‡&call_and_return_all_conditional_losses ˆ__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Č  variables ‚regularization_losses ƒtrainable_variables „ keras_api +‰&call_and_return_all_conditional_losses Š__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_2", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_2", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ą …axis ;gamma <beta † variables ‡regularization_losses ˆtrainable_variables ‰ keras_api +‹&call_and_return_all_conditional_losses Œ__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} – -0 .1 /2 03 14 25 36 47 58 69 710 811 912 :13 ;14 <15" trackable_list_wrapper " trackable_list_wrapper – -0 .1 /2 03 14 25 36 47 58 69 710 811 912 :13 ;14 <15" trackable_list_wrapper ø Š layer_metrics š variables ‹non_trainable_variables Œlayers ›regularization_losses metrics Žlayer_regularization_losses œtrainable_variables ą__call__ +ß&call_and_return_all_conditional_losses 'ß"call_and_return_conditional_losses" _generic_user_object é  _query_dense  _key_dense ‘ _value_dense ’_masked_softmax “_dropout_layer Ÿ _output_dense ” variables •regularization_losses –trainable_variables — keras_api +&call_and_return_all_conditional_losses Ž__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} Ž ˜partial_output_shape ™full_output_shape Ckernel Dbias š variables ›regularization_losses œtrainable_variables  keras_api +&call_and_return_all_conditional_losses __call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Č ž variables Ÿregularization_losses  trainable_variables ” keras_api +‘&call_and_return_all_conditional_losses ’__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_4", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_4", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Š ¢axis Egamma Fbeta £ variables ¤regularization_losses „trainable_variables ¦ keras_api +“&call_and_return_all_conditional_losses ”__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ų Gkernel G_kernel Hbias H_bias § variables Øregularization_losses ©trainable_variables Ŗ keras_api +•&call_and_return_all_conditional_losses –__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ « variables ¬regularization_losses ­trainable_variables ® keras_api +—&call_and_return_all_conditional_losses ˜__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_1", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_1", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Ģ Ikernel I_kernel Jbias J_bias Æ variables °regularization_losses ±trainable_variables ² keras_api +™&call_and_return_all_conditional_losses š__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Č ³ variables “regularization_losses µtrainable_variables ¶ keras_api +›&call_and_return_all_conditional_losses œ__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_5", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_5", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ą ·axis Kgamma Lbeta ø variables ¹regularization_losses ŗtrainable_variables » keras_api +&call_and_return_all_conditional_losses ž__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} – =0 >1 ?2 @3 A4 B5 C6 D7 E8 F9 G10 H11 I12 J13 K14 L15" trackable_list_wrapper " trackable_list_wrapper – =0 >1 ?2 @3 A4 B5 C6 D7 E8 F9 G10 H11 I12 J13 K14 L15" trackable_list_wrapper ø ¼ layer_metrics § variables ½non_trainable_variables ¾layers Øregularization_losses æmetrics Ąlayer_regularization_losses ©trainable_variables ā__call__ +į&call_and_return_all_conditional_losses 'į"call_and_return_conditional_losses" _generic_user_object é Į _query_dense Ā _key_dense Ć _value_dense Ä_masked_softmax Å_dropout_layer ¬ _output_dense Ę variables Ēregularization_losses Čtrainable_variables É keras_api +Ÿ&call_and_return_all_conditional_losses  __call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} Ž Źpartial_output_shape Ėfull_output_shape Skernel Tbias Ģ variables Ķregularization_losses Ītrainable_variables Ļ keras_api +”&call_and_return_all_conditional_losses ¢__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Č Š variables Ńregularization_losses Ņtrainable_variables Ó keras_api +£&call_and_return_all_conditional_losses ¤__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_7", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_7", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Š Ōaxis Ugamma Vbeta Õ variables Öregularization_losses ×trainable_variables Ų keras_api +„&call_and_return_all_conditional_losses ¦__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ų Wkernel W_kernel Xbias X_bias Ł variables Śregularization_losses Ūtrainable_variables Ü keras_api +§&call_and_return_all_conditional_losses Ø__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ Ż variables Žregularization_losses ßtrainable_variables ą keras_api +©&call_and_return_all_conditional_losses Ŗ__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_2", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_2", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Ģ Ykernel Y_kernel Zbias Z_bias į variables āregularization_losses ćtrainable_variables ä keras_api +«&call_and_return_all_conditional_losses ¬__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Č å variables ęregularization_losses ētrainable_variables č keras_api +­&call_and_return_all_conditional_losses ®__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_8", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_8", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ą éaxis [gamma \beta ź variables ėregularization_losses ģtrainable_variables ķ keras_api +Æ&call_and_return_all_conditional_losses °__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} – M0 N1 O2 P3 Q4 R5 S6 T7 U8 V9 W10 X11 Y12 Z13 [14 \15" trackable_list_wrapper " trackable_list_wrapper – M0 N1 O2 P3 Q4 R5 S6 T7 U8 V9 W10 X11 Y12 Z13 [14 \15" trackable_list_wrapper ø ī layer_metrics “ variables ļnon_trainable_variables šlayers µregularization_losses ńmetrics ņlayer_regularization_losses ¶trainable_variables ä__call__ +ć&call_and_return_all_conditional_losses 'ć"call_and_return_conditional_losses" _generic_user_object é ó _query_dense ō _key_dense õ _value_dense ö_masked_softmax ÷_dropout_layer ¹ _output_dense ų variables łregularization_losses śtrainable_variables ū keras_api +±&call_and_return_all_conditional_losses ²__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} Ž üpartial_output_shape żfull_output_shape ckernel dbias ž variables ’regularization_losses €trainable_variables  keras_api +³&call_and_return_all_conditional_losses “__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ‚ variables ƒregularization_losses „trainable_variables … keras_api +µ&call_and_return_all_conditional_losses ¶__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_10", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_10", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Š †axis egamma fbeta ‡ variables ˆregularization_losses ‰trainable_variables Š keras_api +·&call_and_return_all_conditional_losses ø__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ų gkernel g_kernel hbias h_bias ‹ variables Œregularization_losses trainable_variables Ž keras_api +¹&call_and_return_all_conditional_losses ŗ__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½  variables regularization_losses ‘trainable_variables ’ keras_api +»&call_and_return_all_conditional_losses ¼__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_3", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_3", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Ģ ikernel i_kernel jbias j_bias “ variables ”regularization_losses •trainable_variables – keras_api +½&call_and_return_all_conditional_losses ¾__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź — variables ˜regularization_losses ™trainable_variables š keras_api +æ&call_and_return_all_conditional_losses Ą__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_11", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_11", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ą ›axis kgamma lbeta œ variables regularization_losses žtrainable_variables Ÿ keras_api +Į&call_and_return_all_conditional_losses Ā__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} – ]0 ^1 _2 `3 a4 b5 c6 d7 e8 f9 g10 h11 i12 j13 k14 l15" trackable_list_wrapper " trackable_list_wrapper – ]0 ^1 _2 `3 a4 b5 c6 d7 e8 f9 g10 h11 i12 j13 k14 l15" trackable_list_wrapper ø   layer_metrics Į variables ”non_trainable_variables ¢layers Āregularization_losses £metrics ¤layer_regularization_losses Ćtrainable_variables ę__call__ +å&call_and_return_all_conditional_losses 'å"call_and_return_conditional_losses" _generic_user_object é „ _query_dense ¦ _key_dense § _value_dense Ø_masked_softmax ©_dropout_layer Ę _output_dense Ŗ variables «regularization_losses ¬trainable_variables ­ keras_api +Ć&call_and_return_all_conditional_losses Ä__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} Ž ®partial_output_shape Æfull_output_shape skernel tbias ° variables ±regularization_losses ²trainable_variables ³ keras_api +Å&call_and_return_all_conditional_losses Ę__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź “ variables µregularization_losses ¶trainable_variables · keras_api +Ē&call_and_return_all_conditional_losses Č__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_13", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_13", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Š øaxis ugamma vbeta ¹ variables ŗregularization_losses »trainable_variables ¼ keras_api +É&call_and_return_all_conditional_losses Ź__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ų wkernel w_kernel xbias x_bias ½ variables ¾regularization_losses ætrainable_variables Ą keras_api +Ė&call_and_return_all_conditional_losses Ģ__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ Į variables Āregularization_losses Ćtrainable_variables Ä keras_api +Ķ&call_and_return_all_conditional_losses Ī__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_4", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_4", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Ģ ykernel y_kernel zbias z_bias Å variables Ęregularization_losses Ētrainable_variables Č keras_api +Ļ&call_and_return_all_conditional_losses Š__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź É variables Źregularization_losses Ėtrainable_variables Ģ keras_api +Ń&call_and_return_all_conditional_losses Ņ__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_14", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_14", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ą Ķaxis {gamma |beta Ī variables Ļregularization_losses Štrainable_variables Ń keras_api +Ó&call_and_return_all_conditional_losses Ō__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} – m0 n1 o2 p3 q4 r5 s6 t7 u8 v9 w10 x11 y12 z13 {14 |15" trackable_list_wrapper " trackable_list_wrapper – m0 n1 o2 p3 q4 r5 s6 t7 u8 v9 w10 x11 y12 z13 {14 |15" trackable_list_wrapper ø Ņ layer_metrics Ī variables Ónon_trainable_variables Ōlayers Ļregularization_losses Õmetrics Ölayer_regularization_losses Štrainable_variables č__call__ +ē&call_and_return_all_conditional_losses 'ē"call_and_return_conditional_losses" _generic_user_object é × _query_dense Ų _key_dense Ł _value_dense Ś_masked_softmax Ū_dropout_layer Ó _output_dense Ü variables Żregularization_losses Žtrainable_variables ß keras_api +Õ&call_and_return_all_conditional_losses Ö__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą ąpartial_output_shape įfull_output_shape ƒkernel „bias ā variables ćregularization_losses ätrainable_variables å keras_api +×&call_and_return_all_conditional_losses Ų__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ę variables ēregularization_losses čtrainable_variables é keras_api +Ł&call_and_return_all_conditional_losses Ś__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_16", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_16", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ źaxis …gamma †beta ė variables ģregularization_losses ķtrainable_variables ī keras_api +Ū&call_and_return_all_conditional_losses Ü__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü ‡kernel ‡_kernel ˆbias ˆ_bias ļ variables šregularization_losses ńtrainable_variables ņ keras_api +Ż&call_and_return_all_conditional_losses Ž__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ ó variables ōregularization_losses õtrainable_variables ö keras_api +ß&call_and_return_all_conditional_losses ą__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_5", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_5", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š ‰kernel ‰_kernel Šbias Š_bias ÷ variables ųregularization_losses łtrainable_variables ś keras_api +į&call_and_return_all_conditional_losses ā__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź ū variables üregularization_losses żtrainable_variables ž keras_api +ć&call_and_return_all_conditional_losses ä__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_17", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_17", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā ’axis ‹gamma Œbeta € variables regularization_losses ‚trainable_variables ƒ keras_api +å&call_and_return_all_conditional_losses ę__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} £ }0 ~1 2 €3 4 ‚5 ƒ6 „7 …8 †9 ‡10 ˆ11 ‰12 Š13 ‹14 Œ15" trackable_list_wrapper " trackable_list_wrapper £ }0 ~1 2 €3 4 ‚5 ƒ6 „7 …8 †9 ‡10 ˆ11 ‰12 Š13 ‹14 Œ15" trackable_list_wrapper ø „ layer_metrics Ū variables …non_trainable_variables †layers Üregularization_losses ‡metrics ˆlayer_regularization_losses Żtrainable_variables ź__call__ +é&call_and_return_all_conditional_losses 'é"call_and_return_conditional_losses" _generic_user_object é ‰ _query_dense Š _key_dense ‹ _value_dense Œ_masked_softmax _dropout_layer ą _output_dense Ž variables regularization_losses trainable_variables ‘ keras_api +ē&call_and_return_all_conditional_losses č__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą ’partial_output_shape “full_output_shape “kernel ”bias ” variables •regularization_losses –trainable_variables — keras_api +é&call_and_return_all_conditional_losses ź__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ˜ variables ™regularization_losses štrainable_variables › keras_api +ė&call_and_return_all_conditional_losses ģ__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_19", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_19", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ œaxis •gamma –beta  variables žregularization_losses Ÿtrainable_variables   keras_api +ķ&call_and_return_all_conditional_losses ī__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü —kernel —_kernel ˜bias ˜_bias ” variables ¢regularization_losses £trainable_variables ¤ keras_api +ļ&call_and_return_all_conditional_losses š__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ „ variables ¦regularization_losses §trainable_variables Ø keras_api +ń&call_and_return_all_conditional_losses ņ__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_6", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_6", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š ™kernel ™_kernel šbias š_bias © variables Ŗregularization_losses «trainable_variables ¬ keras_api +ó&call_and_return_all_conditional_losses ō__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź ­ variables ®regularization_losses Ætrainable_variables ° keras_api +õ&call_and_return_all_conditional_losses ö__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_20", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_20", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā ±axis ›gamma œbeta ² variables ³regularization_losses “trainable_variables µ keras_api +÷&call_and_return_all_conditional_losses ų__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¦ 0 Ž1 2 3 ‘4 ’5 “6 ”7 •8 –9 —10 ˜11 ™12 š13 ›14 œ15" trackable_list_wrapper " trackable_list_wrapper ¦ 0 Ž1 2 3 ‘4 ’5 “6 ”7 •8 –9 —10 ˜11 ™12 š13 ›14 œ15" trackable_list_wrapper ø ¶ layer_metrics č variables ·non_trainable_variables ølayers éregularization_losses ¹metrics ŗlayer_regularization_losses źtrainable_variables ģ__call__ +ė&call_and_return_all_conditional_losses 'ė"call_and_return_conditional_losses" _generic_user_object é » _query_dense ¼ _key_dense ½ _value_dense ¾_masked_softmax æ_dropout_layer ķ _output_dense Ą variables Įregularization_losses Ātrainable_variables Ć keras_api +ł&call_and_return_all_conditional_losses ś__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą Äpartial_output_shape Åfull_output_shape £kernel ¤bias Ę variables Ēregularization_losses Čtrainable_variables É keras_api +ū&call_and_return_all_conditional_losses ü__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź Ź variables Ėregularization_losses Ģtrainable_variables Ķ keras_api +ż&call_and_return_all_conditional_losses ž__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_22", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_22", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ Īaxis „gamma ¦beta Ļ variables Šregularization_losses Ńtrainable_variables Ņ keras_api +’&call_and_return_all_conditional_losses €__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü §kernel §_kernel Øbias Ø_bias Ó variables Ōregularization_losses Õtrainable_variables Ö keras_api +&call_and_return_all_conditional_losses ‚__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ × variables Ųregularization_losses Łtrainable_variables Ś keras_api +ƒ&call_and_return_all_conditional_losses „__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_7", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_7", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š ©kernel ©_kernel Ŗbias Ŗ_bias Ū variables Üregularization_losses Żtrainable_variables Ž keras_api +…&call_and_return_all_conditional_losses †__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź ß variables ąregularization_losses įtrainable_variables ā keras_api +‡&call_and_return_all_conditional_losses ˆ__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_23", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_23", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā ćaxis «gamma ¬beta ä variables åregularization_losses ętrainable_variables ē keras_api +‰&call_and_return_all_conditional_losses Š__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¦ 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7 „8 ¦9 §10 Ø11 ©12 Ŗ13 «14 ¬15" trackable_list_wrapper " trackable_list_wrapper ¦ 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7 „8 ¦9 §10 Ø11 ©12 Ŗ13 «14 ¬15" trackable_list_wrapper ø č layer_metrics õ variables énon_trainable_variables źlayers öregularization_losses ėmetrics ģlayer_regularization_losses ÷trainable_variables ī__call__ +ķ&call_and_return_all_conditional_losses 'ķ"call_and_return_conditional_losses" _generic_user_object é ķ _query_dense ī _key_dense ļ _value_dense š_masked_softmax ń_dropout_layer ś _output_dense ņ variables óregularization_losses ōtrainable_variables õ keras_api +‹&call_and_return_all_conditional_losses Œ__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą öpartial_output_shape ÷full_output_shape ³kernel “bias ų variables łregularization_losses śtrainable_variables ū keras_api +&call_and_return_all_conditional_losses Ž__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ü variables żregularization_losses žtrainable_variables ’ keras_api +&call_and_return_all_conditional_losses __call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_25", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_25", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ €axis µgamma ¶beta  variables ‚regularization_losses ƒtrainable_variables „ keras_api +‘&call_and_return_all_conditional_losses ’__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü ·kernel ·_kernel øbias ø_bias … variables †regularization_losses ‡trainable_variables ˆ keras_api +“&call_and_return_all_conditional_losses ”__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ ‰ variables Šregularization_losses ‹trainable_variables Œ keras_api +•&call_and_return_all_conditional_losses –__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_8", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_8", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š ¹kernel ¹_kernel ŗbias ŗ_bias  variables Žregularization_losses trainable_variables  keras_api +—&call_and_return_all_conditional_losses ˜__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź ‘ variables ’regularization_losses “trainable_variables ” keras_api +™&call_and_return_all_conditional_losses š__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_26", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_26", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā •axis »gamma ¼beta – variables —regularization_losses ˜trainable_variables ™ keras_api +›&call_and_return_all_conditional_losses œ__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¦ ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7 µ8 ¶9 ·10 ø11 ¹12 ŗ13 »14 ¼15" trackable_list_wrapper " trackable_list_wrapper ¦ ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7 µ8 ¶9 ·10 ø11 ¹12 ŗ13 »14 ¼15" trackable_list_wrapper ø š layer_metrics ‚ variables ›non_trainable_variables œlayers ƒregularization_losses metrics žlayer_regularization_losses „trainable_variables š__call__ +ļ&call_and_return_all_conditional_losses 'ļ"call_and_return_conditional_losses" _generic_user_object é Ÿ _query_dense   _key_dense ” _value_dense ¢_masked_softmax £_dropout_layer ‡ _output_dense ¤ variables „regularization_losses ¦trainable_variables § keras_api +&call_and_return_all_conditional_losses ž__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą Øpartial_output_shape ©full_output_shape Ćkernel Äbias Ŗ variables «regularization_losses ¬trainable_variables ­ keras_api +Ÿ&call_and_return_all_conditional_losses  __call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ® variables Æregularization_losses °trainable_variables ± keras_api +”&call_and_return_all_conditional_losses ¢__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_28", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_28", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ ²axis Ågamma Ębeta ³ variables “regularization_losses µtrainable_variables ¶ keras_api +£&call_and_return_all_conditional_losses ¤__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü Ēkernel Ē_kernel Čbias Č_bias · variables øregularization_losses ¹trainable_variables ŗ keras_api +„&call_and_return_all_conditional_losses ¦__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ½ » variables ¼regularization_losses ½trainable_variables ¾ keras_api +§&call_and_return_all_conditional_losses Ø__call__"Ø _tf_keras_layerŽ{"class_name": "Activation", "name": "activation_9", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_9", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š Ékernel É_kernel Źbias Ź_bias æ variables Ąregularization_losses Įtrainable_variables Ā keras_api +©&call_and_return_all_conditional_losses Ŗ__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź Ć variables Äregularization_losses Åtrainable_variables Ę keras_api +«&call_and_return_all_conditional_losses ¬__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_29", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_29", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā Ēaxis Ėgamma Ģbeta Č variables Éregularization_losses Źtrainable_variables Ė keras_api +­&call_and_return_all_conditional_losses ®__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¦ ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7 Å8 Ę9 Ē10 Č11 É12 Ź13 Ė14 Ģ15" trackable_list_wrapper " trackable_list_wrapper ¦ ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7 Å8 Ę9 Ē10 Č11 É12 Ź13 Ė14 Ģ15" trackable_list_wrapper ø Ģ layer_metrics  variables Ķnon_trainable_variables Īlayers regularization_losses Ļmetrics Šlayer_regularization_losses ‘trainable_variables ņ__call__ +ń&call_and_return_all_conditional_losses 'ń"call_and_return_conditional_losses" _generic_user_object é Ń _query_dense Ņ _key_dense Ó _value_dense Ō_masked_softmax Õ_dropout_layer ” _output_dense Ö variables ×regularization_losses Ųtrainable_variables Ł keras_api +Æ&call_and_return_all_conditional_losses °__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą Śpartial_output_shape Ūfull_output_shape Ókernel Ōbias Ü variables Żregularization_losses Žtrainable_variables ß keras_api +±&call_and_return_all_conditional_losses ²__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ą variables įregularization_losses ātrainable_variables ć keras_api +³&call_and_return_all_conditional_losses “__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_31", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_31", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ äaxis Õgamma Öbeta å variables ęregularization_losses ētrainable_variables č keras_api +µ&call_and_return_all_conditional_losses ¶__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü ×kernel ×_kernel Ųbias Ų_bias é variables źregularization_losses ėtrainable_variables ģ keras_api +·&call_and_return_all_conditional_losses ø__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} æ ķ variables īregularization_losses ļtrainable_variables š keras_api +¹&call_and_return_all_conditional_losses ŗ__call__"Ŗ _tf_keras_layer{"class_name": "Activation", "name": "activation_10", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_10", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š Łkernel Ł_kernel Śbias Ś_bias ń variables ņregularization_losses ótrainable_variables ō keras_api +»&call_and_return_all_conditional_losses ¼__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź õ variables öregularization_losses ÷trainable_variables ų keras_api +½&call_and_return_all_conditional_losses ¾__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_32", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_32", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā łaxis Ūgamma Übeta ś variables ūregularization_losses ütrainable_variables ż keras_api +æ&call_and_return_all_conditional_losses Ą__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¦ Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7 Õ8 Ö9 ×10 Ų11 Ł12 Ś13 Ū14 Ü15" trackable_list_wrapper " trackable_list_wrapper ¦ Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7 Õ8 Ö9 ×10 Ų11 Ł12 Ś13 Ū14 Ü15" trackable_list_wrapper ø ž layer_metrics œ variables ’non_trainable_variables €layers regularization_losses metrics ‚layer_regularization_losses žtrainable_variables ō__call__ +ó&call_and_return_all_conditional_losses 'ó"call_and_return_conditional_losses" _generic_user_object é ƒ _query_dense „ _key_dense … _value_dense †_masked_softmax ‡_dropout_layer ” _output_dense ˆ variables ‰regularization_losses Štrainable_variables ‹ keras_api +Į&call_and_return_all_conditional_losses Ā__call__"Ž _tf_keras_layerÄ{"class_name": "MultiHeadAttention", "name": "self_attention", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention", "trainable": true, "dtype": "float32", "num_heads": 12, "key_size": 64, "value_size": 64, "dropout": 0.1, "use_bias": true, "output_shape": null, "attention_axes": {"class_name": "__tuple__", "items": [1]}, "return_attention_scores": false, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": [{"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}, {"class_name": "TensorShape", "items": [null, null, 768]}]} ą Œpartial_output_shape full_output_shape ćkernel äbias Ž variables regularization_losses trainable_variables ‘ keras_api +Ć&call_and_return_all_conditional_losses Ä__call__"€ _tf_keras_layerę{"class_name": "EinsumDense", "name": "attention_output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "attention_output", "trainable": true, "dtype": "float32", "output_shape": [null, 768], "equation": "abcd,cde->abe", "activation": "linear", "bias_axes": "e", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 12, 64]}} Ź ’ variables “regularization_losses ”trainable_variables • keras_api +Å&call_and_return_all_conditional_losses Ę__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_34", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_34", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ņ –axis ågamma ębeta — variables ˜regularization_losses ™trainable_variables š keras_api +Ē&call_and_return_all_conditional_losses Č__call__"› _tf_keras_layer{"class_name": "LayerNormalization", "name": "self_attention_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "self_attention_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ü ēkernel ē_kernel čbias č_bias › variables œregularization_losses trainable_variables ž keras_api +É&call_and_return_all_conditional_losses Ź__call__"• _tf_keras_layerū{"class_name": "DenseEinsum", "name": "intermediate", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "intermediate", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [3072]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} æ Ÿ variables  regularization_losses ”trainable_variables ¢ keras_api +Ė&call_and_return_all_conditional_losses Ģ__call__"Ŗ _tf_keras_layer{"class_name": "Activation", "name": "activation_11", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "activation_11", "trainable": true, "dtype": "float32", "activation": "Text>gelu"}} Š ékernel é_kernel źbias ź_bias £ variables ¤regularization_losses „trainable_variables ¦ keras_api +Ķ&call_and_return_all_conditional_losses Ī__call__"‰ _tf_keras_layerļ{"class_name": "DenseEinsum", "name": "output", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output", "trainable": true, "dtype": "float32", "output_shape": {"class_name": "__tuple__", "items": [768]}, "num_summed_dimensions": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 3072]}} Ź § variables Øregularization_losses ©trainable_variables Ŗ keras_api +Ļ&call_and_return_all_conditional_losses Š__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_35", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_35", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} Ā «axis ėgamma ģbeta ¬ variables ­regularization_losses ®trainable_variables Æ keras_api +Ń&call_and_return_all_conditional_losses Ņ__call__"‹ _tf_keras_layerń{"class_name": "LayerNormalization", "name": "output_layer_norm", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "output_layer_norm", "trainable": true, "dtype": "float32", "axis": [2], "epsilon": 1e-12, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {}}, "gamma_initializer": {"class_name": "Ones", "config": {}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ¦ Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7 å8 ę9 ē10 č11 é12 ź13 ė14 ģ15" trackable_list_wrapper " trackable_list_wrapper ¦ Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7 å8 ę9 ē10 č11 é12 ź13 ė14 ģ15" trackable_list_wrapper ø ° layer_metrics © variables ±non_trainable_variables ²layers Ŗregularization_losses ³metrics “layer_regularization_losses «trainable_variables ö__call__ +õ&call_and_return_all_conditional_losses 'õ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø µ layer_metrics ­ variables ¶non_trainable_variables ·layers ®regularization_losses ømetrics ¹layer_regularization_losses Ætrainable_variables ų__call__ +÷&call_and_return_all_conditional_losses '÷"call_and_return_conditional_losses" _generic_user_object 0 ķ0 ī1" trackable_list_wrapper " trackable_list_wrapper 0 ķ0 ī1" trackable_list_wrapper ø ŗ layer_metrics ± variables »non_trainable_variables ¼layers ²regularization_losses ½metrics ¾layer_regularization_losses ³trainable_variables ś__call__ +ł&call_and_return_all_conditional_losses 'ł"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper Ö  0  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19  20 !21 "22 #23" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper É æpartial_output_shape Ąfull_output_shape -kernel .bias Į variables Āregularization_losses Ćtrainable_variables Ä keras_api +Ó&call_and_return_all_conditional_losses Ō__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Å Åpartial_output_shape Ęfull_output_shape /kernel 0bias Ē variables Čregularization_losses Étrainable_variables Ź keras_api +Õ&call_and_return_all_conditional_losses Ö__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} É Ėpartial_output_shape Ģfull_output_shape 1kernel 2bias Ķ variables Īregularization_losses Ļtrainable_variables Š keras_api +×&call_and_return_all_conditional_losses Ų__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} ” Ń_mask_expansion_axes Ņ variables Óregularization_losses Ōtrainable_variables Õ keras_api +Ł&call_and_return_all_conditional_losses Ś__call__"ń _tf_keras_layer×{"class_name": "MaskedSoftmax", "name": "masked_softmax", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ä Ö variables ×regularization_losses Ųtrainable_variables Ł keras_api +Ū&call_and_return_all_conditional_losses Ü__call__"Æ _tf_keras_layer•{"class_name": "Dropout", "name": "dropout", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} X -0 .1 /2 03 14 25 36 47" trackable_list_wrapper " trackable_list_wrapper X -0 .1 /2 03 14 25 36 47" trackable_list_wrapper ø Ś layer_metrics ā variables Ūnon_trainable_variables Ülayers ćregularization_losses Żmetrics Žlayer_regularization_losses ätrainable_variables ü__call__ +ū&call_and_return_all_conditional_losses 'ū"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . 30 41" trackable_list_wrapper " trackable_list_wrapper . 30 41" trackable_list_wrapper ø ß layer_metrics č variables ąnon_trainable_variables įlayers éregularization_losses āmetrics ćlayer_regularization_losses źtrainable_variables ž__call__ +ż&call_and_return_all_conditional_losses 'ż"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ä layer_metrics ģ variables ånon_trainable_variables ęlayers ķregularization_losses ēmetrics člayer_regularization_losses ītrainable_variables €__call__ +’&call_and_return_all_conditional_losses '’"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . 50 61" trackable_list_wrapper " trackable_list_wrapper . 50 61" trackable_list_wrapper ø é layer_metrics ń variables źnon_trainable_variables ėlayers ņregularization_losses ģmetrics ķlayer_regularization_losses ótrainable_variables ‚__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object . 70 81" trackable_list_wrapper " trackable_list_wrapper . 70 81" trackable_list_wrapper ø ī layer_metrics õ variables ļnon_trainable_variables šlayers öregularization_losses ńmetrics ņlayer_regularization_losses ÷trainable_variables „__call__ +ƒ&call_and_return_all_conditional_losses 'ƒ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ó layer_metrics ł variables ōnon_trainable_variables õlayers śregularization_losses ömetrics ÷layer_regularization_losses ūtrainable_variables †__call__ +…&call_and_return_all_conditional_losses '…"call_and_return_conditional_losses" _generic_user_object . 90 :1" trackable_list_wrapper " trackable_list_wrapper . 90 :1" trackable_list_wrapper ø ų layer_metrics ż variables łnon_trainable_variables ślayers žregularization_losses ūmetrics ülayer_regularization_losses ’trainable_variables ˆ__call__ +‡&call_and_return_all_conditional_losses '‡"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ż layer_metrics  variables žnon_trainable_variables ’layers ‚regularization_losses € metrics  layer_regularization_losses ƒtrainable_variables Š__call__ +‰&call_and_return_all_conditional_losses '‰"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . ;0 <1" trackable_list_wrapper " trackable_list_wrapper . ;0 <1" trackable_list_wrapper ø ‚  layer_metrics † variables ƒ non_trainable_variables „ layers ‡regularization_losses … metrics † layer_regularization_losses ˆtrainable_variables Œ__call__ +‹&call_and_return_all_conditional_losses '‹"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h ‘0 ’1 “2 ”3 •4 –5 —6 ˜7 ™8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper É ‡ partial_output_shape ˆ full_output_shape =kernel >bias ‰  variables Š regularization_losses ‹ trainable_variables Œ  keras_api +Ż&call_and_return_all_conditional_losses Ž__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Å  partial_output_shape Ž full_output_shape ?kernel @bias   variables  regularization_losses ‘ trainable_variables ’  keras_api +ß&call_and_return_all_conditional_losses ą__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} É “ partial_output_shape ” full_output_shape Akernel Bbias •  variables – regularization_losses — trainable_variables ˜  keras_api +į&call_and_return_all_conditional_losses ā__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ ™ _mask_expansion_axes š  variables › regularization_losses œ trainable_variables   keras_api +ć&call_and_return_all_conditional_losses ä__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_1", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_1", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Č ž  variables Ÿ regularization_losses   trainable_variables ”  keras_api +å&call_and_return_all_conditional_losses ę__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_3", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_3", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} X =0 >1 ?2 @3 A4 B5 C6 D7" trackable_list_wrapper " trackable_list_wrapper X =0 >1 ?2 @3 A4 B5 C6 D7" trackable_list_wrapper ø ¢  layer_metrics ” variables £ non_trainable_variables ¤ layers •regularization_losses „ metrics ¦ layer_regularization_losses –trainable_variables Ž__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . C0 D1" trackable_list_wrapper " trackable_list_wrapper . C0 D1" trackable_list_wrapper ø §  layer_metrics š variables Ø non_trainable_variables © layers ›regularization_losses Ŗ metrics « layer_regularization_losses œtrainable_variables __call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ¬  layer_metrics ž variables ­ non_trainable_variables ® layers Ÿregularization_losses Æ metrics ° layer_regularization_losses  trainable_variables ’__call__ +‘&call_and_return_all_conditional_losses '‘"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . E0 F1" trackable_list_wrapper " trackable_list_wrapper . E0 F1" trackable_list_wrapper ø ±  layer_metrics £ variables ² non_trainable_variables ³ layers ¤regularization_losses “ metrics µ layer_regularization_losses „trainable_variables ”__call__ +“&call_and_return_all_conditional_losses '“"call_and_return_conditional_losses" _generic_user_object . G0 H1" trackable_list_wrapper " trackable_list_wrapper . G0 H1" trackable_list_wrapper ø ¶  layer_metrics § variables · non_trainable_variables ø layers Øregularization_losses ¹ metrics ŗ layer_regularization_losses ©trainable_variables –__call__ +•&call_and_return_all_conditional_losses '•"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø »  layer_metrics « variables ¼ non_trainable_variables ½ layers ¬regularization_losses ¾ metrics æ layer_regularization_losses ­trainable_variables ˜__call__ +—&call_and_return_all_conditional_losses '—"call_and_return_conditional_losses" _generic_user_object . I0 J1" trackable_list_wrapper " trackable_list_wrapper . I0 J1" trackable_list_wrapper ø Ą  layer_metrics Æ variables Į non_trainable_variables Ā layers °regularization_losses Ć metrics Ä layer_regularization_losses ±trainable_variables š__call__ +™&call_and_return_all_conditional_losses '™"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Å  layer_metrics ³ variables Ę non_trainable_variables Ē layers “regularization_losses Č metrics É layer_regularization_losses µtrainable_variables œ__call__ +›&call_and_return_all_conditional_losses '›"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . K0 L1" trackable_list_wrapper " trackable_list_wrapper . K0 L1" trackable_list_wrapper ø Ź  layer_metrics ø variables Ė non_trainable_variables Ģ layers ¹regularization_losses Ķ metrics Ī layer_regularization_losses ŗtrainable_variables ž__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h ž0 Ÿ1  2 ”3 ¢4 £5 ¤6 „7 ¦8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper É Ļ partial_output_shape Š full_output_shape Mkernel Nbias Ń  variables Ņ regularization_losses Ó trainable_variables Ō  keras_api +ē&call_and_return_all_conditional_losses č__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Å Õ partial_output_shape Ö full_output_shape Okernel Pbias ×  variables Ų regularization_losses Ł trainable_variables Ś  keras_api +é&call_and_return_all_conditional_losses ź__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} É Ū partial_output_shape Ü full_output_shape Qkernel Rbias Ż  variables Ž regularization_losses ß trainable_variables ą  keras_api +ė&call_and_return_all_conditional_losses ģ__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ į _mask_expansion_axes ā  variables ć regularization_losses ä trainable_variables å  keras_api +ķ&call_and_return_all_conditional_losses ī__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_2", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_2", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Č ę  variables ē regularization_losses č trainable_variables é  keras_api +ļ&call_and_return_all_conditional_losses š__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_6", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_6", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} X M0 N1 O2 P3 Q4 R5 S6 T7" trackable_list_wrapper " trackable_list_wrapper X M0 N1 O2 P3 Q4 R5 S6 T7" trackable_list_wrapper ø ź  layer_metrics Ę variables ė non_trainable_variables ģ layers Ēregularization_losses ķ metrics ī layer_regularization_losses Čtrainable_variables  __call__ +Ÿ&call_and_return_all_conditional_losses 'Ÿ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . S0 T1" trackable_list_wrapper " trackable_list_wrapper . S0 T1" trackable_list_wrapper ø ļ  layer_metrics Ģ variables š non_trainable_variables ń layers Ķregularization_losses ņ metrics ó layer_regularization_losses Ītrainable_variables ¢__call__ +”&call_and_return_all_conditional_losses '”"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ō  layer_metrics Š variables õ non_trainable_variables ö layers Ńregularization_losses ÷ metrics ų layer_regularization_losses Ņtrainable_variables ¤__call__ +£&call_and_return_all_conditional_losses '£"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . U0 V1" trackable_list_wrapper " trackable_list_wrapper . U0 V1" trackable_list_wrapper ø ł  layer_metrics Õ variables ś non_trainable_variables ū layers Öregularization_losses ü metrics ż layer_regularization_losses ×trainable_variables ¦__call__ +„&call_and_return_all_conditional_losses '„"call_and_return_conditional_losses" _generic_user_object . W0 X1" trackable_list_wrapper " trackable_list_wrapper . W0 X1" trackable_list_wrapper ø ž  layer_metrics Ł variables ’ non_trainable_variables € layers Śregularization_losses  metrics ‚ layer_regularization_losses Ūtrainable_variables Ø__call__ +§&call_and_return_all_conditional_losses '§"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ƒ  layer_metrics Ż variables „ non_trainable_variables … layers Žregularization_losses † metrics ‡ layer_regularization_losses ßtrainable_variables Ŗ__call__ +©&call_and_return_all_conditional_losses '©"call_and_return_conditional_losses" _generic_user_object . Y0 Z1" trackable_list_wrapper " trackable_list_wrapper . Y0 Z1" trackable_list_wrapper ø ˆ  layer_metrics į variables ‰ non_trainable_variables Š layers āregularization_losses ‹ metrics Œ layer_regularization_losses ćtrainable_variables ¬__call__ +«&call_and_return_all_conditional_losses '«"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø   layer_metrics å variables Ž non_trainable_variables  layers ęregularization_losses  metrics ‘ layer_regularization_losses ētrainable_variables ®__call__ +­&call_and_return_all_conditional_losses '­"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . [0 \1" trackable_list_wrapper " trackable_list_wrapper . [0 \1" trackable_list_wrapper ø ’  layer_metrics ź variables “ non_trainable_variables ” layers ėregularization_losses • metrics – layer_regularization_losses ģtrainable_variables °__call__ +Æ&call_and_return_all_conditional_losses 'Æ"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h «0 ¬1 ­2 ®3 Æ4 °5 ±6 ²7 ³8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper É — partial_output_shape ˜ full_output_shape ]kernel ^bias ™  variables š regularization_losses › trainable_variables œ  keras_api +ń&call_and_return_all_conditional_losses ņ__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Å  partial_output_shape ž full_output_shape _kernel `bias Ÿ  variables   regularization_losses ” trainable_variables ¢  keras_api +ó&call_and_return_all_conditional_losses ō__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} É £ partial_output_shape ¤ full_output_shape akernel bbias „  variables ¦ regularization_losses § trainable_variables Ø  keras_api +õ&call_and_return_all_conditional_losses ö__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ © _mask_expansion_axes Ŗ  variables « regularization_losses ¬ trainable_variables ­  keras_api +÷&call_and_return_all_conditional_losses ų__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_3", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_3", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Č ®  variables Æ regularization_losses ° trainable_variables ±  keras_api +ł&call_and_return_all_conditional_losses ś__call__"³ _tf_keras_layer™{"class_name": "Dropout", "name": "dropout_9", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_9", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} X ]0 ^1 _2 `3 a4 b5 c6 d7" trackable_list_wrapper " trackable_list_wrapper X ]0 ^1 _2 `3 a4 b5 c6 d7" trackable_list_wrapper ø ²  layer_metrics ų variables ³ non_trainable_variables “ layers łregularization_losses µ metrics ¶ layer_regularization_losses śtrainable_variables ²__call__ +±&call_and_return_all_conditional_losses '±"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . c0 d1" trackable_list_wrapper " trackable_list_wrapper . c0 d1" trackable_list_wrapper ø ·  layer_metrics ž variables ø non_trainable_variables ¹ layers ’regularization_losses ŗ metrics » layer_regularization_losses €trainable_variables “__call__ +³&call_and_return_all_conditional_losses '³"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ¼  layer_metrics ‚ variables ½ non_trainable_variables ¾ layers ƒregularization_losses æ metrics Ą layer_regularization_losses „trainable_variables ¶__call__ +µ&call_and_return_all_conditional_losses 'µ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . e0 f1" trackable_list_wrapper " trackable_list_wrapper . e0 f1" trackable_list_wrapper ø Į  layer_metrics ‡ variables Ā non_trainable_variables Ć layers ˆregularization_losses Ä metrics Å layer_regularization_losses ‰trainable_variables ø__call__ +·&call_and_return_all_conditional_losses '·"call_and_return_conditional_losses" _generic_user_object . g0 h1" trackable_list_wrapper " trackable_list_wrapper . g0 h1" trackable_list_wrapper ø Ę  layer_metrics ‹ variables Ē non_trainable_variables Č layers Œregularization_losses É metrics Ź layer_regularization_losses trainable_variables ŗ__call__ +¹&call_and_return_all_conditional_losses '¹"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ė  layer_metrics  variables Ģ non_trainable_variables Ķ layers regularization_losses Ī metrics Ļ layer_regularization_losses ‘trainable_variables ¼__call__ +»&call_and_return_all_conditional_losses '»"call_and_return_conditional_losses" _generic_user_object . i0 j1" trackable_list_wrapper " trackable_list_wrapper . i0 j1" trackable_list_wrapper ø Š  layer_metrics “ variables Ń non_trainable_variables Ņ layers ”regularization_losses Ó metrics Ō layer_regularization_losses •trainable_variables ¾__call__ +½&call_and_return_all_conditional_losses '½"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Õ  layer_metrics — variables Ö non_trainable_variables × layers ˜regularization_losses Ų metrics Ł layer_regularization_losses ™trainable_variables Ą__call__ +æ&call_and_return_all_conditional_losses 'æ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . k0 l1" trackable_list_wrapper " trackable_list_wrapper . k0 l1" trackable_list_wrapper ø Ś  layer_metrics œ variables Ū non_trainable_variables Ü layers regularization_losses Ż metrics Ž layer_regularization_losses žtrainable_variables Ā__call__ +Į&call_and_return_all_conditional_losses 'Į"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h ø0 ¹1 ŗ2 »3 ¼4 ½5 ¾6 æ7 Ą8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper É ß partial_output_shape ą full_output_shape mkernel nbias į  variables ā regularization_losses ć trainable_variables ä  keras_api +ū&call_and_return_all_conditional_losses ü__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Å å partial_output_shape ę full_output_shape okernel pbias ē  variables č regularization_losses é trainable_variables ź  keras_api +ż&call_and_return_all_conditional_losses ž__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} É ė partial_output_shape ģ full_output_shape qkernel rbias ķ  variables ī regularization_losses ļ trainable_variables š  keras_api +’&call_and_return_all_conditional_losses €__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ ń _mask_expansion_axes ņ  variables ó regularization_losses ō trainable_variables õ  keras_api +&call_and_return_all_conditional_losses ‚__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_4", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_4", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź ö  variables ÷ regularization_losses ų trainable_variables ł  keras_api +ƒ&call_and_return_all_conditional_losses „__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_12", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_12", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} X m0 n1 o2 p3 q4 r5 s6 t7" trackable_list_wrapper " trackable_list_wrapper X m0 n1 o2 p3 q4 r5 s6 t7" trackable_list_wrapper ø ś  layer_metrics Ŗ variables ū non_trainable_variables ü layers «regularization_losses ż metrics ž layer_regularization_losses ¬trainable_variables Ä__call__ +Ć&call_and_return_all_conditional_losses 'Ć"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . s0 t1" trackable_list_wrapper " trackable_list_wrapper . s0 t1" trackable_list_wrapper ø ’  layer_metrics ° variables € non_trainable_variables  layers ±regularization_losses ‚ metrics ƒ layer_regularization_losses ²trainable_variables Ę__call__ +Å&call_and_return_all_conditional_losses 'Å"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø „  layer_metrics “ variables … non_trainable_variables † layers µregularization_losses ‡ metrics ˆ layer_regularization_losses ¶trainable_variables Č__call__ +Ē&call_and_return_all_conditional_losses 'Ē"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . u0 v1" trackable_list_wrapper " trackable_list_wrapper . u0 v1" trackable_list_wrapper ø ‰  layer_metrics ¹ variables Š non_trainable_variables ‹ layers ŗregularization_losses Œ metrics  layer_regularization_losses »trainable_variables Ź__call__ +É&call_and_return_all_conditional_losses 'É"call_and_return_conditional_losses" _generic_user_object . w0 x1" trackable_list_wrapper " trackable_list_wrapper . w0 x1" trackable_list_wrapper ø Ž  layer_metrics ½ variables  non_trainable_variables  layers ¾regularization_losses ‘ metrics ’ layer_regularization_losses ætrainable_variables Ģ__call__ +Ė&call_and_return_all_conditional_losses 'Ė"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø “  layer_metrics Į variables ” non_trainable_variables • layers Āregularization_losses – metrics — layer_regularization_losses Ćtrainable_variables Ī__call__ +Ķ&call_and_return_all_conditional_losses 'Ķ"call_and_return_conditional_losses" _generic_user_object . y0 z1" trackable_list_wrapper " trackable_list_wrapper . y0 z1" trackable_list_wrapper ø ˜  layer_metrics Å variables ™ non_trainable_variables š layers Ęregularization_losses › metrics œ layer_regularization_losses Ētrainable_variables Š__call__ +Ļ&call_and_return_all_conditional_losses 'Ļ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø   layer_metrics É variables ž non_trainable_variables Ÿ layers Źregularization_losses   metrics ” layer_regularization_losses Ėtrainable_variables Ņ__call__ +Ń&call_and_return_all_conditional_losses 'Ń"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper . {0 |1" trackable_list_wrapper " trackable_list_wrapper . {0 |1" trackable_list_wrapper ø ¢  layer_metrics Ī variables £ non_trainable_variables ¤ layers Ļregularization_losses „ metrics ¦ layer_regularization_losses Štrainable_variables Ō__call__ +Ó&call_and_return_all_conditional_losses 'Ó"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h Å0 Ę1 Ē2 Č3 É4 Ź5 Ė6 Ģ7 Ķ8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper É § partial_output_shape Ø full_output_shape }kernel ~bias ©  variables Ŗ regularization_losses « trainable_variables ¬  keras_api +…&call_and_return_all_conditional_losses †__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ę ­ partial_output_shape ® full_output_shape kernel €bias Æ  variables ° regularization_losses ± trainable_variables ²  keras_api +‡&call_and_return_all_conditional_losses ˆ__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė ³ partial_output_shape “ full_output_shape kernel ‚bias µ  variables ¶ regularization_losses · trainable_variables ø  keras_api +‰&call_and_return_all_conditional_losses Š__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ ¹ _mask_expansion_axes ŗ  variables » regularization_losses ¼ trainable_variables ½  keras_api +‹&call_and_return_all_conditional_losses Œ__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_5", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_5", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź ¾  variables æ regularization_losses Ą trainable_variables Į  keras_api +&call_and_return_all_conditional_losses Ž__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_15", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_15", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ] }0 ~1 2 €3 4 ‚5 ƒ6 „7" trackable_list_wrapper " trackable_list_wrapper ] }0 ~1 2 €3 4 ‚5 ƒ6 „7" trackable_list_wrapper ø Ā  layer_metrics Ü variables Ć non_trainable_variables Ä layers Żregularization_losses Å metrics Ę layer_regularization_losses Žtrainable_variables Ö__call__ +Õ&call_and_return_all_conditional_losses 'Õ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ƒ0 „1" trackable_list_wrapper " trackable_list_wrapper 0 ƒ0 „1" trackable_list_wrapper ø Ē  layer_metrics ā variables Č non_trainable_variables É layers ćregularization_losses Ź metrics Ė layer_regularization_losses ätrainable_variables Ų__call__ +×&call_and_return_all_conditional_losses '×"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ģ  layer_metrics ę variables Ķ non_trainable_variables Ī layers ēregularization_losses Ļ metrics Š layer_regularization_losses čtrainable_variables Ś__call__ +Ł&call_and_return_all_conditional_losses 'Ł"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 …0 †1" trackable_list_wrapper " trackable_list_wrapper 0 …0 †1" trackable_list_wrapper ø Ń  layer_metrics ė variables Ņ non_trainable_variables Ó layers ģregularization_losses Ō metrics Õ layer_regularization_losses ķtrainable_variables Ü__call__ +Ū&call_and_return_all_conditional_losses 'Ū"call_and_return_conditional_losses" _generic_user_object 0 ‡0 ˆ1" trackable_list_wrapper " trackable_list_wrapper 0 ‡0 ˆ1" trackable_list_wrapper ø Ö  layer_metrics ļ variables × non_trainable_variables Ų layers šregularization_losses Ł metrics Ś layer_regularization_losses ńtrainable_variables Ž__call__ +Ż&call_and_return_all_conditional_losses 'Ż"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ū  layer_metrics ó variables Ü non_trainable_variables Ż layers ōregularization_losses Ž metrics ß layer_regularization_losses õtrainable_variables ą__call__ +ß&call_and_return_all_conditional_losses 'ß"call_and_return_conditional_losses" _generic_user_object 0 ‰0 Š1" trackable_list_wrapper " trackable_list_wrapper 0 ‰0 Š1" trackable_list_wrapper ø ą  layer_metrics ÷ variables į non_trainable_variables ā layers ųregularization_losses ć metrics ä layer_regularization_losses łtrainable_variables ā__call__ +į&call_and_return_all_conditional_losses 'į"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø å  layer_metrics ū variables ę non_trainable_variables ē layers üregularization_losses č metrics é layer_regularization_losses żtrainable_variables ä__call__ +ć&call_and_return_all_conditional_losses 'ć"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 ‹0 Œ1" trackable_list_wrapper " trackable_list_wrapper 0 ‹0 Œ1" trackable_list_wrapper ø ź  layer_metrics € variables ė non_trainable_variables ģ layers regularization_losses ķ metrics ī layer_regularization_losses ‚trainable_variables ę__call__ +å&call_and_return_all_conditional_losses 'å"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h Ņ0 Ó1 Ō2 Õ3 Ö4 ×5 Ų6 Ł7 Ś8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper Ė ļ partial_output_shape š full_output_shape kernel Žbias ń  variables ņ regularization_losses ó trainable_variables ō  keras_api +&call_and_return_all_conditional_losses __call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ē õ partial_output_shape ö full_output_shape kernel bias ÷  variables ų regularization_losses ł trainable_variables ś  keras_api +‘&call_and_return_all_conditional_losses ’__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė ū partial_output_shape ü full_output_shape ‘kernel ’bias ż  variables ž regularization_losses ’ trainable_variables €  keras_api +“&call_and_return_all_conditional_losses ”__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „  _mask_expansion_axes ‚  variables ƒ regularization_losses „ trainable_variables …  keras_api +•&call_and_return_all_conditional_losses –__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_6", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_6", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź †  variables ‡ regularization_losses ˆ trainable_variables ‰  keras_api +—&call_and_return_all_conditional_losses ˜__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_18", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_18", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ` 0 Ž1 2 3 ‘4 ’5 “6 ”7" trackable_list_wrapper " trackable_list_wrapper ` 0 Ž1 2 3 ‘4 ’5 “6 ”7" trackable_list_wrapper ø Š  layer_metrics Ž variables ‹ non_trainable_variables Œ layers regularization_losses  metrics Ž layer_regularization_losses trainable_variables č__call__ +ē&call_and_return_all_conditional_losses 'ē"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 “0 ”1" trackable_list_wrapper " trackable_list_wrapper 0 “0 ”1" trackable_list_wrapper ø   layer_metrics ” variables  non_trainable_variables ‘ layers •regularization_losses ’ metrics “ layer_regularization_losses –trainable_variables ź__call__ +é&call_and_return_all_conditional_losses 'é"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ”  layer_metrics ˜ variables • non_trainable_variables – layers ™regularization_losses — metrics ˜ layer_regularization_losses štrainable_variables ģ__call__ +ė&call_and_return_all_conditional_losses 'ė"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 •0 –1" trackable_list_wrapper " trackable_list_wrapper 0 •0 –1" trackable_list_wrapper ø ™  layer_metrics  variables š non_trainable_variables › layers žregularization_losses œ metrics  layer_regularization_losses Ÿtrainable_variables ī__call__ +ķ&call_and_return_all_conditional_losses 'ķ"call_and_return_conditional_losses" _generic_user_object 0 —0 ˜1" trackable_list_wrapper " trackable_list_wrapper 0 —0 ˜1" trackable_list_wrapper ø ž  layer_metrics ” variables Ÿ non_trainable_variables   layers ¢regularization_losses ” metrics ¢ layer_regularization_losses £trainable_variables š__call__ +ļ&call_and_return_all_conditional_losses 'ļ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø £  layer_metrics „ variables ¤ non_trainable_variables „ layers ¦regularization_losses ¦ metrics § layer_regularization_losses §trainable_variables ņ__call__ +ń&call_and_return_all_conditional_losses 'ń"call_and_return_conditional_losses" _generic_user_object 0 ™0 š1" trackable_list_wrapper " trackable_list_wrapper 0 ™0 š1" trackable_list_wrapper ø Ø  layer_metrics © variables © non_trainable_variables Ŗ layers Ŗregularization_losses « metrics ¬ layer_regularization_losses «trainable_variables ō__call__ +ó&call_and_return_all_conditional_losses 'ó"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ­  layer_metrics ­ variables ® non_trainable_variables Æ layers ®regularization_losses ° metrics ± layer_regularization_losses Ætrainable_variables ö__call__ +õ&call_and_return_all_conditional_losses 'õ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 ›0 œ1" trackable_list_wrapper " trackable_list_wrapper 0 ›0 œ1" trackable_list_wrapper ø ²  layer_metrics ² variables ³ non_trainable_variables “ layers ³regularization_losses µ metrics ¶ layer_regularization_losses “trainable_variables ų__call__ +÷&call_and_return_all_conditional_losses '÷"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h ß0 ą1 į2 ā3 ć4 ä5 å6 ę7 ē8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper Ė · partial_output_shape ø full_output_shape kernel žbias ¹  variables ŗ regularization_losses » trainable_variables ¼  keras_api +™&call_and_return_all_conditional_losses š__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ē ½ partial_output_shape ¾ full_output_shape Ÿkernel  bias æ  variables Ą regularization_losses Į trainable_variables Ā  keras_api +›&call_and_return_all_conditional_losses œ__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė Ć partial_output_shape Ä full_output_shape ”kernel ¢bias Å  variables Ę regularization_losses Ē trainable_variables Č  keras_api +&call_and_return_all_conditional_losses ž__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ É _mask_expansion_axes Ź  variables Ė regularization_losses Ģ trainable_variables Ķ  keras_api +Ÿ&call_and_return_all_conditional_losses  __call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_7", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_7", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź Ī  variables Ļ regularization_losses Š trainable_variables Ń  keras_api +”&call_and_return_all_conditional_losses ¢__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_21", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_21", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ` 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7" trackable_list_wrapper " trackable_list_wrapper ` 0 ž1 Ÿ2  3 ”4 ¢5 £6 ¤7" trackable_list_wrapper ø Ņ  layer_metrics Ą variables Ó non_trainable_variables Ō layers Įregularization_losses Õ metrics Ö layer_regularization_losses Ātrainable_variables ś__call__ +ł&call_and_return_all_conditional_losses 'ł"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 £0 ¤1" trackable_list_wrapper " trackable_list_wrapper 0 £0 ¤1" trackable_list_wrapper ø ×  layer_metrics Ę variables Ų non_trainable_variables Ł layers Ēregularization_losses Ś metrics Ū layer_regularization_losses Čtrainable_variables ü__call__ +ū&call_and_return_all_conditional_losses 'ū"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ü  layer_metrics Ź variables Ż non_trainable_variables Ž layers Ėregularization_losses ß metrics ą layer_regularization_losses Ģtrainable_variables ž__call__ +ż&call_and_return_all_conditional_losses 'ż"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 „0 ¦1" trackable_list_wrapper " trackable_list_wrapper 0 „0 ¦1" trackable_list_wrapper ø į  layer_metrics Ļ variables ā non_trainable_variables ć layers Šregularization_losses ä metrics å layer_regularization_losses Ńtrainable_variables €__call__ +’&call_and_return_all_conditional_losses '’"call_and_return_conditional_losses" _generic_user_object 0 §0 Ø1" trackable_list_wrapper " trackable_list_wrapper 0 §0 Ø1" trackable_list_wrapper ø ę  layer_metrics Ó variables ē non_trainable_variables č layers Ōregularization_losses é metrics ź layer_regularization_losses Õtrainable_variables ‚__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ė  layer_metrics × variables ģ non_trainable_variables ķ layers Ųregularization_losses ī metrics ļ layer_regularization_losses Łtrainable_variables „__call__ +ƒ&call_and_return_all_conditional_losses 'ƒ"call_and_return_conditional_losses" _generic_user_object 0 ©0 Ŗ1" trackable_list_wrapper " trackable_list_wrapper 0 ©0 Ŗ1" trackable_list_wrapper ø š  layer_metrics Ū variables ń non_trainable_variables ņ layers Üregularization_losses ó metrics ō layer_regularization_losses Żtrainable_variables †__call__ +…&call_and_return_all_conditional_losses '…"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø õ  layer_metrics ß variables ö non_trainable_variables ÷ layers ąregularization_losses ų metrics ł layer_regularization_losses įtrainable_variables ˆ__call__ +‡&call_and_return_all_conditional_losses '‡"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 «0 ¬1" trackable_list_wrapper " trackable_list_wrapper 0 «0 ¬1" trackable_list_wrapper ø ś  layer_metrics ä variables ū non_trainable_variables ü layers åregularization_losses ż metrics ž layer_regularization_losses ętrainable_variables Š__call__ +‰&call_and_return_all_conditional_losses '‰"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h ģ0 ķ1 ī2 ļ3 š4 ń5 ņ6 ó7 ō8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper Ė ’ partial_output_shape € full_output_shape ­kernel ®bias   variables ‚ regularization_losses ƒ trainable_variables „  keras_api +£&call_and_return_all_conditional_losses ¤__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ē … partial_output_shape † full_output_shape Ækernel °bias ‡  variables ˆ regularization_losses ‰ trainable_variables Š  keras_api +„&call_and_return_all_conditional_losses ¦__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė ‹ partial_output_shape Œ full_output_shape ±kernel ²bias   variables Ž regularization_losses  trainable_variables   keras_api +§&call_and_return_all_conditional_losses Ø__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ ‘ _mask_expansion_axes ’  variables “ regularization_losses ” trainable_variables •  keras_api +©&call_and_return_all_conditional_losses Ŗ__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_8", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_8", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź –  variables — regularization_losses ˜ trainable_variables ™  keras_api +«&call_and_return_all_conditional_losses ¬__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_24", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_24", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ` ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7" trackable_list_wrapper " trackable_list_wrapper ` ­0 ®1 Æ2 °3 ±4 ²5 ³6 “7" trackable_list_wrapper ø š  layer_metrics ņ variables › non_trainable_variables œ layers óregularization_losses  metrics ž layer_regularization_losses ōtrainable_variables Œ__call__ +‹&call_and_return_all_conditional_losses '‹"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ³0 “1" trackable_list_wrapper " trackable_list_wrapper 0 ³0 “1" trackable_list_wrapper ø Ÿ  layer_metrics ų variables   non_trainable_variables ” layers łregularization_losses ¢ metrics £ layer_regularization_losses śtrainable_variables Ž__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ¤  layer_metrics ü variables „ non_trainable_variables ¦ layers żregularization_losses § metrics Ø layer_regularization_losses žtrainable_variables __call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 µ0 ¶1" trackable_list_wrapper " trackable_list_wrapper 0 µ0 ¶1" trackable_list_wrapper ø ©  layer_metrics  variables Ŗ non_trainable_variables « layers ‚regularization_losses ¬ metrics ­ layer_regularization_losses ƒtrainable_variables ’__call__ +‘&call_and_return_all_conditional_losses '‘"call_and_return_conditional_losses" _generic_user_object 0 ·0 ø1" trackable_list_wrapper " trackable_list_wrapper 0 ·0 ø1" trackable_list_wrapper ø ®  layer_metrics … variables Æ non_trainable_variables ° layers †regularization_losses ± metrics ² layer_regularization_losses ‡trainable_variables ”__call__ +“&call_and_return_all_conditional_losses '“"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ³  layer_metrics ‰ variables “ non_trainable_variables µ layers Šregularization_losses ¶ metrics · layer_regularization_losses ‹trainable_variables –__call__ +•&call_and_return_all_conditional_losses '•"call_and_return_conditional_losses" _generic_user_object 0 ¹0 ŗ1" trackable_list_wrapper " trackable_list_wrapper 0 ¹0 ŗ1" trackable_list_wrapper ø ø  layer_metrics  variables ¹ non_trainable_variables ŗ layers Žregularization_losses » metrics ¼ layer_regularization_losses trainable_variables ˜__call__ +—&call_and_return_all_conditional_losses '—"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ½  layer_metrics ‘ variables ¾ non_trainable_variables æ layers ’regularization_losses Ą metrics Į layer_regularization_losses “trainable_variables š__call__ +™&call_and_return_all_conditional_losses '™"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 »0 ¼1" trackable_list_wrapper " trackable_list_wrapper 0 »0 ¼1" trackable_list_wrapper ø Ā  layer_metrics – variables Ć non_trainable_variables Ä layers —regularization_losses Å metrics Ę layer_regularization_losses ˜trainable_variables œ__call__ +›&call_and_return_all_conditional_losses '›"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h ł0 ś1 ū2 ü3 ż4 ž5 ’6 €7 8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper Ė Ē partial_output_shape Č full_output_shape ½kernel ¾bias É  variables Ź regularization_losses Ė trainable_variables Ģ  keras_api +­&call_and_return_all_conditional_losses ®__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ē Ķ partial_output_shape Ī full_output_shape ækernel Ąbias Ļ  variables Š regularization_losses Ń trainable_variables Ņ  keras_api +Æ&call_and_return_all_conditional_losses °__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė Ó partial_output_shape Ō full_output_shape Įkernel Ābias Õ  variables Ö regularization_losses × trainable_variables Ų  keras_api +±&call_and_return_all_conditional_losses ²__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} „ Ł _mask_expansion_axes Ś  variables Ū regularization_losses Ü trainable_variables Ż  keras_api +³&call_and_return_all_conditional_losses “__call__"õ _tf_keras_layerŪ{"class_name": "MaskedSoftmax", "name": "masked_softmax_9", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_9", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź Ž  variables ß regularization_losses ą trainable_variables į  keras_api +µ&call_and_return_all_conditional_losses ¶__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_27", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_27", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ` ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7" trackable_list_wrapper " trackable_list_wrapper ` ½0 ¾1 æ2 Ą3 Į4 Ā5 Ć6 Ä7" trackable_list_wrapper ø ā  layer_metrics ¤ variables ć non_trainable_variables ä layers „regularization_losses å metrics ę layer_regularization_losses ¦trainable_variables ž__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Ć0 Ä1" trackable_list_wrapper " trackable_list_wrapper 0 Ć0 Ä1" trackable_list_wrapper ø ē  layer_metrics Ŗ variables č non_trainable_variables é layers «regularization_losses ź metrics ė layer_regularization_losses ¬trainable_variables  __call__ +Ÿ&call_and_return_all_conditional_losses 'Ÿ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ģ  layer_metrics ® variables ķ non_trainable_variables ī layers Æregularization_losses ļ metrics š layer_regularization_losses °trainable_variables ¢__call__ +”&call_and_return_all_conditional_losses '”"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 Å0 Ę1" trackable_list_wrapper " trackable_list_wrapper 0 Å0 Ę1" trackable_list_wrapper ø ń  layer_metrics ³ variables ņ non_trainable_variables ó layers “regularization_losses ō metrics õ layer_regularization_losses µtrainable_variables ¤__call__ +£&call_and_return_all_conditional_losses '£"call_and_return_conditional_losses" _generic_user_object 0 Ē0 Č1" trackable_list_wrapper " trackable_list_wrapper 0 Ē0 Č1" trackable_list_wrapper ø ö  layer_metrics · variables ÷ non_trainable_variables ų layers øregularization_losses ł metrics ś layer_regularization_losses ¹trainable_variables ¦__call__ +„&call_and_return_all_conditional_losses '„"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ū  layer_metrics » variables ü non_trainable_variables ż layers ¼regularization_losses ž metrics ’ layer_regularization_losses ½trainable_variables Ø__call__ +§&call_and_return_all_conditional_losses '§"call_and_return_conditional_losses" _generic_user_object 0 É0 Ź1" trackable_list_wrapper " trackable_list_wrapper 0 É0 Ź1" trackable_list_wrapper ø € layer_metrics æ variables non_trainable_variables ‚layers Ąregularization_losses ƒmetrics „layer_regularization_losses Įtrainable_variables Ŗ__call__ +©&call_and_return_all_conditional_losses '©"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø … layer_metrics Ć variables †non_trainable_variables ‡layers Äregularization_losses ˆmetrics ‰layer_regularization_losses Åtrainable_variables ¬__call__ +«&call_and_return_all_conditional_losses '«"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 Ė0 Ģ1" trackable_list_wrapper " trackable_list_wrapper 0 Ė0 Ģ1" trackable_list_wrapper ø Š layer_metrics Č variables ‹non_trainable_variables Œlayers Éregularization_losses metrics Žlayer_regularization_losses Źtrainable_variables ®__call__ +­&call_and_return_all_conditional_losses '­"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h †0 ‡1 ˆ2 ‰3 Š4 ‹5 Œ6 7 Ž8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper Ė partial_output_shape full_output_shape Ķkernel Ībias ‘ variables ’regularization_losses “trainable_variables ” keras_api +·&call_and_return_all_conditional_losses ø__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ē •partial_output_shape –full_output_shape Ļkernel Šbias — variables ˜regularization_losses ™trainable_variables š keras_api +¹&call_and_return_all_conditional_losses ŗ__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė ›partial_output_shape œfull_output_shape Ńkernel Ņbias  variables žregularization_losses Ÿtrainable_variables   keras_api +»&call_and_return_all_conditional_losses ¼__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} § ”_mask_expansion_axes ¢ variables £regularization_losses ¤trainable_variables „ keras_api +½&call_and_return_all_conditional_losses ¾__call__"÷ _tf_keras_layerŻ{"class_name": "MaskedSoftmax", "name": "masked_softmax_10", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_10", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź ¦ variables §regularization_losses Øtrainable_variables © keras_api +æ&call_and_return_all_conditional_losses Ą__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_30", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_30", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ` Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7" trackable_list_wrapper " trackable_list_wrapper ` Ķ0 Ī1 Ļ2 Š3 Ń4 Ņ5 Ó6 Ō7" trackable_list_wrapper ø Ŗ layer_metrics Ö variables «non_trainable_variables ¬layers ×regularization_losses ­metrics ®layer_regularization_losses Ųtrainable_variables °__call__ +Æ&call_and_return_all_conditional_losses 'Æ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Ó0 Ō1" trackable_list_wrapper " trackable_list_wrapper 0 Ó0 Ō1" trackable_list_wrapper ø Æ layer_metrics Ü variables °non_trainable_variables ±layers Żregularization_losses ²metrics ³layer_regularization_losses Žtrainable_variables ²__call__ +±&call_and_return_all_conditional_losses '±"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø “ layer_metrics ą variables µnon_trainable_variables ¶layers įregularization_losses ·metrics ølayer_regularization_losses ātrainable_variables “__call__ +³&call_and_return_all_conditional_losses '³"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 Õ0 Ö1" trackable_list_wrapper " trackable_list_wrapper 0 Õ0 Ö1" trackable_list_wrapper ø ¹ layer_metrics å variables ŗnon_trainable_variables »layers ęregularization_losses ¼metrics ½layer_regularization_losses ētrainable_variables ¶__call__ +µ&call_and_return_all_conditional_losses 'µ"call_and_return_conditional_losses" _generic_user_object 0 ×0 Ų1" trackable_list_wrapper " trackable_list_wrapper 0 ×0 Ų1" trackable_list_wrapper ø ¾ layer_metrics é variables ænon_trainable_variables Ąlayers źregularization_losses Įmetrics Ālayer_regularization_losses ėtrainable_variables ø__call__ +·&call_and_return_all_conditional_losses '·"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ć layer_metrics ķ variables Änon_trainable_variables Ålayers īregularization_losses Ęmetrics Ēlayer_regularization_losses ļtrainable_variables ŗ__call__ +¹&call_and_return_all_conditional_losses '¹"call_and_return_conditional_losses" _generic_user_object 0 Ł0 Ś1" trackable_list_wrapper " trackable_list_wrapper 0 Ł0 Ś1" trackable_list_wrapper ø Č layer_metrics ń variables Énon_trainable_variables Źlayers ņregularization_losses Ėmetrics Ģlayer_regularization_losses ótrainable_variables ¼__call__ +»&call_and_return_all_conditional_losses '»"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ķ layer_metrics õ variables Īnon_trainable_variables Ļlayers öregularization_losses Šmetrics Ńlayer_regularization_losses ÷trainable_variables ¾__call__ +½&call_and_return_all_conditional_losses '½"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 Ū0 Ü1" trackable_list_wrapper " trackable_list_wrapper 0 Ū0 Ü1" trackable_list_wrapper ø Ņ layer_metrics ś variables Ónon_trainable_variables Ōlayers ūregularization_losses Õmetrics Ölayer_regularization_losses ütrainable_variables Ą__call__ +æ&call_and_return_all_conditional_losses 'æ"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h “0 ”1 •2 –3 —4 ˜5 ™6 š7 ›8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper Ė ×partial_output_shape Ųfull_output_shape Żkernel Žbias Ł variables Śregularization_losses Ūtrainable_variables Ü keras_api +Į&call_and_return_all_conditional_losses Ā__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "query", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "query", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ē Żpartial_output_shape Žfull_output_shape ßkernel ąbias ß variables ąregularization_losses įtrainable_variables ā keras_api +Ć&call_and_return_all_conditional_losses Ä__call__"ē _tf_keras_layerĶ{"class_name": "EinsumDense", "name": "key", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "key", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} Ė ćpartial_output_shape äfull_output_shape įkernel ābias å variables ęregularization_losses ētrainable_variables č keras_api +Å&call_and_return_all_conditional_losses Ę__call__"ė _tf_keras_layerŃ{"class_name": "EinsumDense", "name": "value", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "value", "trainable": true, "dtype": "float32", "output_shape": [null, 12, 64], "equation": "abc,cde->abde", "activation": "linear", "bias_axes": "de", "kernel_initializer": {"class_name": "TruncatedNormal", "config": {"mean": 0.0, "stddev": 0.02, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "build_input_shape": {"class_name": "TensorShape", "items": [null, null, 768]}} § é_mask_expansion_axes ź variables ėregularization_losses ģtrainable_variables ķ keras_api +Ē&call_and_return_all_conditional_losses Č__call__"÷ _tf_keras_layerŻ{"class_name": "MaskedSoftmax", "name": "masked_softmax_11", "trainable": true, "expects_training_arg": false, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "masked_softmax_11", "trainable": true, "dtype": "float32", "mask_expansion_axes": [1], "normalization_axes": {"class_name": "__tuple__", "items": [3]}}} Ź ī variables ļregularization_losses štrainable_variables ń keras_api +É&call_and_return_all_conditional_losses Ź__call__"µ _tf_keras_layer›{"class_name": "Dropout", "name": "dropout_33", "trainable": true, "expects_training_arg": true, "dtype": "float32", "batch_input_shape": null, "stateful": false, "config": {"name": "dropout_33", "trainable": true, "dtype": "float32", "rate": 0.1, "noise_shape": null, "seed": null}} ` Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7" trackable_list_wrapper " trackable_list_wrapper ` Ż0 Ž1 ß2 ą3 į4 ā5 ć6 ä7" trackable_list_wrapper ø ņ layer_metrics ˆ variables ónon_trainable_variables ōlayers ‰regularization_losses õmetrics ölayer_regularization_losses Štrainable_variables Ā__call__ +Į&call_and_return_all_conditional_losses 'Į"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ć0 ä1" trackable_list_wrapper " trackable_list_wrapper 0 ć0 ä1" trackable_list_wrapper ø ÷ layer_metrics Ž variables ųnon_trainable_variables łlayers regularization_losses śmetrics ūlayer_regularization_losses trainable_variables Ä__call__ +Ć&call_and_return_all_conditional_losses 'Ć"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ü layer_metrics ’ variables żnon_trainable_variables žlayers “regularization_losses ’metrics €layer_regularization_losses ”trainable_variables Ę__call__ +Å&call_and_return_all_conditional_losses 'Å"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 å0 ę1" trackable_list_wrapper " trackable_list_wrapper 0 å0 ę1" trackable_list_wrapper ø  layer_metrics — variables ‚non_trainable_variables ƒlayers ˜regularization_losses „metrics …layer_regularization_losses ™trainable_variables Č__call__ +Ē&call_and_return_all_conditional_losses 'Ē"call_and_return_conditional_losses" _generic_user_object 0 ē0 č1" trackable_list_wrapper " trackable_list_wrapper 0 ē0 č1" trackable_list_wrapper ø † layer_metrics › variables ‡non_trainable_variables ˆlayers œregularization_losses ‰metrics Šlayer_regularization_losses trainable_variables Ź__call__ +É&call_and_return_all_conditional_losses 'É"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ‹ layer_metrics Ÿ variables Œnon_trainable_variables layers  regularization_losses Žmetrics layer_regularization_losses ”trainable_variables Ģ__call__ +Ė&call_and_return_all_conditional_losses 'Ė"call_and_return_conditional_losses" _generic_user_object 0 é0 ź1" trackable_list_wrapper " trackable_list_wrapper 0 é0 ź1" trackable_list_wrapper ø  layer_metrics £ variables ‘non_trainable_variables ’layers ¤regularization_losses “metrics ”layer_regularization_losses „trainable_variables Ī__call__ +Ķ&call_and_return_all_conditional_losses 'Ķ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø • layer_metrics § variables –non_trainable_variables —layers Øregularization_losses ˜metrics ™layer_regularization_losses ©trainable_variables Š__call__ +Ļ&call_and_return_all_conditional_losses 'Ļ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper 0 ė0 ģ1" trackable_list_wrapper " trackable_list_wrapper 0 ė0 ģ1" trackable_list_wrapper ø š layer_metrics ¬ variables ›non_trainable_variables œlayers ­regularization_losses metrics žlayer_regularization_losses ®trainable_variables Ņ__call__ +Ń&call_and_return_all_conditional_losses 'Ń"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper h  0 ”1 ¢2 £3 ¤4 „5 ¦6 §7 Ø8" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper . -0 .1" trackable_list_wrapper " trackable_list_wrapper . -0 .1" trackable_list_wrapper ø Ÿ layer_metrics Į variables  non_trainable_variables ”layers Āregularization_losses ¢metrics £layer_regularization_losses Ćtrainable_variables Ō__call__ +Ó&call_and_return_all_conditional_losses 'Ó"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . /0 01" trackable_list_wrapper " trackable_list_wrapper . /0 01" trackable_list_wrapper ø ¤ layer_metrics Ē variables „non_trainable_variables ¦layers Čregularization_losses §metrics Ølayer_regularization_losses Étrainable_variables Ö__call__ +Õ&call_and_return_all_conditional_losses 'Õ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . 10 21" trackable_list_wrapper " trackable_list_wrapper . 10 21" trackable_list_wrapper ø © layer_metrics Ķ variables Ŗnon_trainable_variables «layers Īregularization_losses ¬metrics ­layer_regularization_losses Ļtrainable_variables Ų__call__ +×&call_and_return_all_conditional_losses '×"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ® layer_metrics Ņ variables Ænon_trainable_variables °layers Óregularization_losses ±metrics ²layer_regularization_losses Ōtrainable_variables Ś__call__ +Ł&call_and_return_all_conditional_losses 'Ł"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ³ layer_metrics Ö variables “non_trainable_variables µlayers ×regularization_losses ¶metrics ·layer_regularization_losses Ųtrainable_variables Ü__call__ +Ū&call_and_return_all_conditional_losses 'Ū"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P Ż0 Ž1 ß2 ą3 į4 ’5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper . =0 >1" trackable_list_wrapper " trackable_list_wrapper . =0 >1" trackable_list_wrapper ø ø layer_metrics ‰  variables ¹non_trainable_variables ŗlayers Š regularization_losses »metrics ¼layer_regularization_losses ‹ trainable_variables Ž__call__ +Ż&call_and_return_all_conditional_losses 'Ż"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . ?0 @1" trackable_list_wrapper " trackable_list_wrapper . ?0 @1" trackable_list_wrapper ø ½ layer_metrics   variables ¾non_trainable_variables ælayers  regularization_losses Ąmetrics Įlayer_regularization_losses ‘ trainable_variables ą__call__ +ß&call_and_return_all_conditional_losses 'ß"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . A0 B1" trackable_list_wrapper " trackable_list_wrapper . A0 B1" trackable_list_wrapper ø Ā layer_metrics •  variables Ćnon_trainable_variables Älayers – regularization_losses Åmetrics Ęlayer_regularization_losses — trainable_variables ā__call__ +į&call_and_return_all_conditional_losses 'į"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ē layer_metrics š  variables Čnon_trainable_variables Élayers › regularization_losses Źmetrics Ėlayer_regularization_losses œ trainable_variables ä__call__ +ć&call_and_return_all_conditional_losses 'ć"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ģ layer_metrics ž  variables Ķnon_trainable_variables Īlayers Ÿ regularization_losses Ļmetrics Šlayer_regularization_losses   trainable_variables ę__call__ +å&call_and_return_all_conditional_losses 'å"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P 0 1 ‘2 ’3 “4 Ÿ5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper . M0 N1" trackable_list_wrapper " trackable_list_wrapper . M0 N1" trackable_list_wrapper ø Ń layer_metrics Ń  variables Ņnon_trainable_variables Ólayers Ņ regularization_losses Ōmetrics Õlayer_regularization_losses Ó trainable_variables č__call__ +ē&call_and_return_all_conditional_losses 'ē"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . O0 P1" trackable_list_wrapper " trackable_list_wrapper . O0 P1" trackable_list_wrapper ø Ö layer_metrics ×  variables ×non_trainable_variables Ųlayers Ų regularization_losses Łmetrics Ślayer_regularization_losses Ł trainable_variables ź__call__ +é&call_and_return_all_conditional_losses 'é"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . Q0 R1" trackable_list_wrapper " trackable_list_wrapper . Q0 R1" trackable_list_wrapper ø Ū layer_metrics Ż  variables Ünon_trainable_variables Żlayers Ž regularization_losses Žmetrics ßlayer_regularization_losses ß trainable_variables ģ__call__ +ė&call_and_return_all_conditional_losses 'ė"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ą layer_metrics ā  variables įnon_trainable_variables ālayers ć regularization_losses ćmetrics älayer_regularization_losses ä trainable_variables ī__call__ +ķ&call_and_return_all_conditional_losses 'ķ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø å layer_metrics ę  variables ęnon_trainable_variables ēlayers ē regularization_losses čmetrics élayer_regularization_losses č trainable_variables š__call__ +ļ&call_and_return_all_conditional_losses 'ļ"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P Į0 Ā1 Ć2 Ä3 Å4 ¬5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper . ]0 ^1" trackable_list_wrapper " trackable_list_wrapper . ]0 ^1" trackable_list_wrapper ø ź layer_metrics ™  variables ėnon_trainable_variables ģlayers š regularization_losses ķmetrics īlayer_regularization_losses › trainable_variables ņ__call__ +ń&call_and_return_all_conditional_losses 'ń"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . _0 `1" trackable_list_wrapper " trackable_list_wrapper . _0 `1" trackable_list_wrapper ø ļ layer_metrics Ÿ  variables šnon_trainable_variables ńlayers   regularization_losses ņmetrics ólayer_regularization_losses ” trainable_variables ō__call__ +ó&call_and_return_all_conditional_losses 'ó"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . a0 b1" trackable_list_wrapper " trackable_list_wrapper . a0 b1" trackable_list_wrapper ø ō layer_metrics „  variables õnon_trainable_variables ölayers ¦ regularization_losses ÷metrics ųlayer_regularization_losses § trainable_variables ö__call__ +õ&call_and_return_all_conditional_losses 'õ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ł layer_metrics Ŗ  variables śnon_trainable_variables ūlayers « regularization_losses ümetrics żlayer_regularization_losses ¬ trainable_variables ų__call__ +÷&call_and_return_all_conditional_losses '÷"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ž layer_metrics ®  variables ’non_trainable_variables €layers Æ regularization_losses metrics ‚layer_regularization_losses ° trainable_variables ś__call__ +ł&call_and_return_all_conditional_losses 'ł"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P ó0 ō1 õ2 ö3 ÷4 ¹5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper . m0 n1" trackable_list_wrapper " trackable_list_wrapper . m0 n1" trackable_list_wrapper ø ƒ layer_metrics į  variables „non_trainable_variables …layers ā regularization_losses †metrics ‡layer_regularization_losses ć trainable_variables ü__call__ +ū&call_and_return_all_conditional_losses 'ū"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . o0 p1" trackable_list_wrapper " trackable_list_wrapper . o0 p1" trackable_list_wrapper ø ˆ layer_metrics ē  variables ‰non_trainable_variables Šlayers č regularization_losses ‹metrics Œlayer_regularization_losses é trainable_variables ž__call__ +ż&call_and_return_all_conditional_losses 'ż"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper . q0 r1" trackable_list_wrapper " trackable_list_wrapper . q0 r1" trackable_list_wrapper ø  layer_metrics ķ  variables Žnon_trainable_variables layers ī regularization_losses metrics ‘layer_regularization_losses ļ trainable_variables €__call__ +’&call_and_return_all_conditional_losses '’"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ’ layer_metrics ņ  variables “non_trainable_variables ”layers ó regularization_losses •metrics –layer_regularization_losses ō trainable_variables ‚__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø — layer_metrics ö  variables ˜non_trainable_variables ™layers ÷ regularization_losses šmetrics ›layer_regularization_losses ų trainable_variables „__call__ +ƒ&call_and_return_all_conditional_losses 'ƒ"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P „0 ¦1 §2 Ø3 ©4 Ę5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper . }0 ~1" trackable_list_wrapper " trackable_list_wrapper . }0 ~1" trackable_list_wrapper ø œ layer_metrics ©  variables non_trainable_variables žlayers Ŗ regularization_losses Ÿmetrics  layer_regularization_losses « trainable_variables †__call__ +…&call_and_return_all_conditional_losses '…"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper / 0 €1" trackable_list_wrapper " trackable_list_wrapper / 0 €1" trackable_list_wrapper ø ” layer_metrics Æ  variables ¢non_trainable_variables £layers ° regularization_losses ¤metrics „layer_regularization_losses ± trainable_variables ˆ__call__ +‡&call_and_return_all_conditional_losses '‡"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 0 ‚1" trackable_list_wrapper " trackable_list_wrapper 0 0 ‚1" trackable_list_wrapper ø ¦ layer_metrics µ  variables §non_trainable_variables Ølayers ¶ regularization_losses ©metrics Ŗlayer_regularization_losses · trainable_variables Š__call__ +‰&call_and_return_all_conditional_losses '‰"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø « layer_metrics ŗ  variables ¬non_trainable_variables ­layers » regularization_losses ®metrics Ælayer_regularization_losses ¼ trainable_variables Œ__call__ +‹&call_and_return_all_conditional_losses '‹"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ° layer_metrics ¾  variables ±non_trainable_variables ²layers æ regularization_losses ³metrics “layer_regularization_losses Ą trainable_variables Ž__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P ×0 Ų1 Ł2 Ś3 Ū4 Ó5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper 0 0 Ž1" trackable_list_wrapper " trackable_list_wrapper 0 0 Ž1" trackable_list_wrapper ø µ layer_metrics ń  variables ¶non_trainable_variables ·layers ņ regularization_losses ømetrics ¹layer_regularization_losses ó trainable_variables __call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 0 1" trackable_list_wrapper " trackable_list_wrapper 0 0 1" trackable_list_wrapper ø ŗ layer_metrics ÷  variables »non_trainable_variables ¼layers ų regularization_losses ½metrics ¾layer_regularization_losses ł trainable_variables ’__call__ +‘&call_and_return_all_conditional_losses '‘"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ‘0 ’1" trackable_list_wrapper " trackable_list_wrapper 0 ‘0 ’1" trackable_list_wrapper ø æ layer_metrics ż  variables Ąnon_trainable_variables Įlayers ž regularization_losses Āmetrics Ćlayer_regularization_losses ’ trainable_variables ”__call__ +“&call_and_return_all_conditional_losses '“"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ä layer_metrics ‚  variables Ånon_trainable_variables Ęlayers ƒ regularization_losses Ēmetrics Člayer_regularization_losses „ trainable_variables –__call__ +•&call_and_return_all_conditional_losses '•"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø É layer_metrics †  variables Źnon_trainable_variables Ėlayers ‡ regularization_losses Ģmetrics Ķlayer_regularization_losses ˆ trainable_variables ˜__call__ +—&call_and_return_all_conditional_losses '—"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P ‰0 Š1 ‹2 Œ3 4 ą5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper 0 0 ž1" trackable_list_wrapper " trackable_list_wrapper 0 0 ž1" trackable_list_wrapper ø Ī layer_metrics ¹  variables Ļnon_trainable_variables Šlayers ŗ regularization_losses Ńmetrics Ņlayer_regularization_losses » trainable_variables š__call__ +™&call_and_return_all_conditional_losses '™"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Ÿ0  1" trackable_list_wrapper " trackable_list_wrapper 0 Ÿ0  1" trackable_list_wrapper ø Ó layer_metrics æ  variables Ōnon_trainable_variables Õlayers Ą regularization_losses Ömetrics ×layer_regularization_losses Į trainable_variables œ__call__ +›&call_and_return_all_conditional_losses '›"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ”0 ¢1" trackable_list_wrapper " trackable_list_wrapper 0 ”0 ¢1" trackable_list_wrapper ø Ų layer_metrics Å  variables Łnon_trainable_variables Ślayers Ę regularization_losses Ūmetrics Ülayer_regularization_losses Ē trainable_variables ž__call__ +&call_and_return_all_conditional_losses '"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ż layer_metrics Ź  variables Žnon_trainable_variables ßlayers Ė regularization_losses ąmetrics įlayer_regularization_losses Ģ trainable_variables  __call__ +Ÿ&call_and_return_all_conditional_losses 'Ÿ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ā layer_metrics Ī  variables ćnon_trainable_variables älayers Ļ regularization_losses åmetrics ęlayer_regularization_losses Š trainable_variables ¢__call__ +”&call_and_return_all_conditional_losses '”"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P »0 ¼1 ½2 ¾3 æ4 ķ5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper 0 ­0 ®1" trackable_list_wrapper " trackable_list_wrapper 0 ­0 ®1" trackable_list_wrapper ø ē layer_metrics   variables čnon_trainable_variables élayers ‚ regularization_losses źmetrics ėlayer_regularization_losses ƒ trainable_variables ¤__call__ +£&call_and_return_all_conditional_losses '£"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Æ0 °1" trackable_list_wrapper " trackable_list_wrapper 0 Æ0 °1" trackable_list_wrapper ø ģ layer_metrics ‡  variables ķnon_trainable_variables īlayers ˆ regularization_losses ļmetrics šlayer_regularization_losses ‰ trainable_variables ¦__call__ +„&call_and_return_all_conditional_losses '„"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ±0 ²1" trackable_list_wrapper " trackable_list_wrapper 0 ±0 ²1" trackable_list_wrapper ø ń layer_metrics   variables ņnon_trainable_variables ólayers Ž regularization_losses ōmetrics õlayer_regularization_losses  trainable_variables Ø__call__ +§&call_and_return_all_conditional_losses '§"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ö layer_metrics ’  variables ÷non_trainable_variables ųlayers “ regularization_losses łmetrics ślayer_regularization_losses ” trainable_variables Ŗ__call__ +©&call_and_return_all_conditional_losses '©"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ū layer_metrics –  variables ünon_trainable_variables żlayers — regularization_losses žmetrics ’layer_regularization_losses ˜ trainable_variables ¬__call__ +«&call_and_return_all_conditional_losses '«"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P ķ0 ī1 ļ2 š3 ń4 ś5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper 0 ½0 ¾1" trackable_list_wrapper " trackable_list_wrapper 0 ½0 ¾1" trackable_list_wrapper ø € layer_metrics É  variables non_trainable_variables ‚layers Ź regularization_losses ƒmetrics „layer_regularization_losses Ė trainable_variables ®__call__ +­&call_and_return_all_conditional_losses '­"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 æ0 Ą1" trackable_list_wrapper " trackable_list_wrapper 0 æ0 Ą1" trackable_list_wrapper ø … layer_metrics Ļ  variables †non_trainable_variables ‡layers Š regularization_losses ˆmetrics ‰layer_regularization_losses Ń trainable_variables °__call__ +Æ&call_and_return_all_conditional_losses 'Æ"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Į0 Ā1" trackable_list_wrapper " trackable_list_wrapper 0 Į0 Ā1" trackable_list_wrapper ø Š layer_metrics Õ  variables ‹non_trainable_variables Œlayers Ö regularization_losses metrics Žlayer_regularization_losses × trainable_variables ²__call__ +±&call_and_return_all_conditional_losses '±"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø  layer_metrics Ś  variables non_trainable_variables ‘layers Ū regularization_losses ’metrics “layer_regularization_losses Ü trainable_variables “__call__ +³&call_and_return_all_conditional_losses '³"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ” layer_metrics Ž  variables •non_trainable_variables –layers ß regularization_losses —metrics ˜layer_regularization_losses ą trainable_variables ¶__call__ +µ&call_and_return_all_conditional_losses 'µ"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P Ÿ0  1 ”2 ¢3 £4 ‡5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper 0 Ķ0 Ī1" trackable_list_wrapper " trackable_list_wrapper 0 Ķ0 Ī1" trackable_list_wrapper ø ™ layer_metrics ‘ variables šnon_trainable_variables ›layers ’regularization_losses œmetrics layer_regularization_losses “trainable_variables ø__call__ +·&call_and_return_all_conditional_losses '·"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Ļ0 Š1" trackable_list_wrapper " trackable_list_wrapper 0 Ļ0 Š1" trackable_list_wrapper ø ž layer_metrics — variables Ÿnon_trainable_variables  layers ˜regularization_losses ”metrics ¢layer_regularization_losses ™trainable_variables ŗ__call__ +¹&call_and_return_all_conditional_losses '¹"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 Ń0 Ņ1" trackable_list_wrapper " trackable_list_wrapper 0 Ń0 Ņ1" trackable_list_wrapper ø £ layer_metrics  variables ¤non_trainable_variables „layers žregularization_losses ¦metrics §layer_regularization_losses Ÿtrainable_variables ¼__call__ +»&call_and_return_all_conditional_losses '»"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ø layer_metrics ¢ variables ©non_trainable_variables Ŗlayers £regularization_losses «metrics ¬layer_regularization_losses ¤trainable_variables ¾__call__ +½&call_and_return_all_conditional_losses '½"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø ­ layer_metrics ¦ variables ®non_trainable_variables Ælayers §regularization_losses °metrics ±layer_regularization_losses Øtrainable_variables Ą__call__ +æ&call_and_return_all_conditional_losses 'æ"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P Ń0 Ņ1 Ó2 Ō3 Õ4 ”5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper 0 Ż0 Ž1" trackable_list_wrapper " trackable_list_wrapper 0 Ż0 Ž1" trackable_list_wrapper ø ² layer_metrics Ł variables ³non_trainable_variables “layers Śregularization_losses µmetrics ¶layer_regularization_losses Ūtrainable_variables Ā__call__ +Į&call_and_return_all_conditional_losses 'Į"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 ß0 ą1" trackable_list_wrapper " trackable_list_wrapper 0 ß0 ą1" trackable_list_wrapper ø · layer_metrics ß variables ønon_trainable_variables ¹layers ąregularization_losses ŗmetrics »layer_regularization_losses įtrainable_variables Ä__call__ +Ć&call_and_return_all_conditional_losses 'Ć"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper 0 į0 ā1" trackable_list_wrapper " trackable_list_wrapper 0 į0 ā1" trackable_list_wrapper ø ¼ layer_metrics å variables ½non_trainable_variables ¾layers ęregularization_losses æmetrics Ąlayer_regularization_losses ētrainable_variables Ę__call__ +Å&call_and_return_all_conditional_losses 'Å"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Į layer_metrics ź variables Ānon_trainable_variables Ćlayers ėregularization_losses Ämetrics Ålayer_regularization_losses ģtrainable_variables Č__call__ +Ē&call_and_return_all_conditional_losses 'Ē"call_and_return_conditional_losses" _generic_user_object " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ø Ę layer_metrics ī variables Ēnon_trainable_variables Člayers ļregularization_losses Émetrics Źlayer_regularization_losses štrainable_variables Ź__call__ +É&call_and_return_all_conditional_losses 'É"call_and_return_conditional_losses" _generic_user_object " trackable_dict_wrapper " trackable_list_wrapper P ƒ0 „1 …2 †3 ‡4 ”5" trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_dict_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper " trackable_list_wrapper ź2ē G__inference_functional_1_layer_call_and_return_conditional_losses_15364 G__inference_functional_1_layer_call_and_return_conditional_losses_20746 G__inference_functional_1_layer_call_and_return_conditional_losses_19407 G__inference_functional_1_layer_call_and_return_conditional_losses_15771Ą ·²³ FullArgSpec1 args)š& jself jinputs jtraining jmask varargs  varkw  defaults š p   kwonlyargsš kwonlydefaultsŖ annotationsŖ* ž2ū ,__inference_functional_1_layer_call_fn_17400 ,__inference_functional_1_layer_call_fn_16586 ,__inference_functional_1_layer_call_fn_21560 ,__inference_functional_1_layer_call_fn_21153Ą ·²³ FullArgSpec1 args)š& jself jinputs jtraining jmask varargs  varkw  defaults š p   kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ų2Õ __inference__wrapped_model_7608± ‹²‡ FullArgSpec argsš varargsjargs varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ¢œ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ †2ƒ N__inference_transformer_encoder_layer_call_and_return_conditional_losses_23158 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12475 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12049 N__inference_transformer_encoder_layer_call_and_return_conditional_losses_24497Ą ·²³ FullArgSpec1 args)š& jself jinputs jtraining jmask varargs  varkw  defaults š p   kwonlyargsš kwonlydefaultsŖ annotationsŖ* š2— 3__inference_transformer_encoder_layer_call_fn_24904 3__inference_transformer_encoder_layer_call_fn_25311 3__inference_transformer_encoder_layer_call_fn_14142 3__inference_transformer_encoder_layer_call_fn_13309Ą ·²³ FullArgSpec1 args)š& jself jinputs jtraining jmask varargs  varkw  defaults š p   kwonlyargsš kwonlydefaultsŖ annotationsŖ* SBQ #__inference_signature_wrapper_17809 input_maskinput_type_idsinput_word_ids ō2ń J__inference_word_embeddings_layer_call_and_return_conditional_losses_25325¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö /__inference_word_embeddings_layer_call_fn_25332¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ÷2ō M__inference_position_embedding_layer_call_and_return_conditional_losses_25355¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ü2Ł 2__inference_position_embedding_layer_call_fn_25362¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ō2ń J__inference_type_embeddings_layer_call_and_return_conditional_losses_25380¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö /__inference_type_embeddings_layer_call_fn_25387¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* č2å >__inference_add_layer_call_and_return_conditional_losses_25395¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ķ2Ź #__inference_add_layer_call_fn_25402¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ś2÷ P__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_25424¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ß2Ü 5__inference_embeddings/layer_norm_layer_call_fn_25433¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ā2æ B__inference_dropout_layer_call_and_return_conditional_losses_25445 B__inference_dropout_layer_call_and_return_conditional_losses_25450“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Œ2‰ '__inference_dropout_layer_call_fn_25460 '__inference_dropout_layer_call_fn_25455“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ų2õ N__inference_self_attention_mask_layer_call_and_return_conditional_losses_25497¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ż2Ś 3__inference_self_attention_mask_layer_call_fn_25503¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_0_layer_call_and_return_conditional_losses_25631 N__inference_transformer/layer_0_layer_call_and_return_conditional_losses_25738³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_0_layer_call_fn_25814 3__inference_transformer/layer_0_layer_call_fn_25776³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_1_layer_call_and_return_conditional_losses_26049 N__inference_transformer/layer_1_layer_call_and_return_conditional_losses_25942³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_1_layer_call_fn_26125 3__inference_transformer/layer_1_layer_call_fn_26087³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_2_layer_call_and_return_conditional_losses_26360 N__inference_transformer/layer_2_layer_call_and_return_conditional_losses_26253³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_2_layer_call_fn_26398 3__inference_transformer/layer_2_layer_call_fn_26436³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_3_layer_call_and_return_conditional_losses_26564 N__inference_transformer/layer_3_layer_call_and_return_conditional_losses_26671³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_3_layer_call_fn_26709 3__inference_transformer/layer_3_layer_call_fn_26747³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_4_layer_call_and_return_conditional_losses_26875 N__inference_transformer/layer_4_layer_call_and_return_conditional_losses_26982³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_4_layer_call_fn_27020 3__inference_transformer/layer_4_layer_call_fn_27058³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_5_layer_call_and_return_conditional_losses_27186 N__inference_transformer/layer_5_layer_call_and_return_conditional_losses_27293³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_5_layer_call_fn_27331 3__inference_transformer/layer_5_layer_call_fn_27369³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_27497 N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_27604³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_6_layer_call_fn_27680 3__inference_transformer/layer_6_layer_call_fn_27642³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_27808 N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_27915³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_7_layer_call_fn_27991 3__inference_transformer/layer_7_layer_call_fn_27953³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_28226 N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_28119³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_8_layer_call_fn_28264 3__inference_transformer/layer_8_layer_call_fn_28302³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ł2Ö N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_28430 N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_28537³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* £2  3__inference_transformer/layer_9_layer_call_fn_28613 3__inference_transformer/layer_9_layer_call_fn_28575³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ū2Ų O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_28741 O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_28848³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* „2¢ 4__inference_transformer/layer_10_layer_call_fn_28886 4__inference_transformer/layer_10_layer_call_fn_28924³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ū2Ų O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_29052 O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_29159³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* „2¢ 4__inference_transformer/layer_11_layer_call_fn_29235 4__inference_transformer/layer_11_layer_call_fn_29197³ Ŗ²¦ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaults  annotationsŖ* Ģ2É A__inference_lambda_layer_call_and_return_conditional_losses_29244 A__inference_lambda_layer_call_and_return_conditional_losses_29253Ą ·²³ FullArgSpec1 args)š& jself jinputs jmask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaultsŖ annotationsŖ* –2“ &__inference_lambda_layer_call_fn_29258 &__inference_lambda_layer_call_fn_29263Ą ·²³ FullArgSpec1 args)š& jself jinputs jmask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaultsŖ annotationsŖ* õ2ņ K__inference_pooler_transform_layer_call_and_return_conditional_losses_29274¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ś2× 0__inference_pooler_transform_layer_call_fn_29283¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ļ2ĢÉ Ą²¼ FullArgSpec; args3š0 jself jinputs jattention_mask jtraining varargs  varkw  defaults š  p kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* Ø2„¢ ™²• FullArgSpec argsš jself jinputs varargs  varkw  defaults  kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* µ2²Æ ¦²¢ FullArgSpec% argsš jself jscores jmask varargs  varkw  defaults¢   kwonlyargsš kwonlydefaults  annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ŗ2·“ «²§ FullArgSpec) args!š jself jinputs jtraining varargs  varkw  defaultsš p kwonlyargsš kwonlydefaultsŖ annotationsŖ* ² __inference__wrapped_model_7608޶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī¬¢Ø  ¢œ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ Ŗ"£ŖŸ E transformer_encoder.Š+ transformer_encoder ’’’’’’’’’€ V transformer_encoder_1=Š: transformer_encoder_1 ’’’’’’’’’ ’’’’’’’’’€Ø >__inference_add_layer_call_and_return_conditional_losses_25395å­¢© ”¢ šš– 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 0Š- inputs/1 ’’’’’’’’’ ’’’’’’’’’€ 0Š- inputs/2 ’’’’’’’’’ ’’’’’’’’’€ Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š€ #__inference_add_layer_call_fn_25402Ų­¢© ”¢ šš– 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 0Š- inputs/1 ’’’’’’’’’ ’’’’’’’’’€ 0Š- inputs/2 ’’’’’’’’’ ’’’’’’’’’€ Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€¾ B__inference_dropout_layer_call_and_return_conditional_losses_25445xA¢> 7¢4 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š¾ B__inference_dropout_layer_call_and_return_conditional_losses_25450xA¢> 7¢4 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š– '__inference_dropout_layer_call_fn_25455kA¢> 7¢4 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€– '__inference_dropout_layer_call_fn_25460kA¢> 7¢4 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ģ P__inference_embeddings/layer_norm_layer_call_and_return_conditional_losses_25424x+,=¢: 3¢0 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š¤ 5__inference_embeddings/layer_norm_layer_call_fn_25433k+,=¢: 3¢0 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€˜ G__inference_functional_1_layer_call_and_return_conditional_losses_15364̶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM Š 0/0 ’’’’’’’’’€ +Š( 0/1 ’’’’’’’’’ ’’’’’’’’’€ š˜ G__inference_functional_1_layer_call_and_return_conditional_losses_15771̶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM Š 0/0 ’’’’’’’’’€ +Š( 0/1 ’’’’’’’’’ ’’’’’’’’’€ šŠ G__inference_functional_1_layer_call_and_return_conditional_losses_19407¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM Š 0/0 ’’’’’’’’’€ +Š( 0/1 ’’’’’’’’’ ’’’’’’’’’€ šŠ G__inference_functional_1_layer_call_and_return_conditional_losses_20746¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM Š 0/0 ’’’’’’’’’€ +Š( 0/1 ’’’’’’’’’ ’’’’’’’’’€ šļ ,__inference_functional_1_layer_call_fn_16586¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI Š 0 ’’’’’’’’’€ )Š& 1 ’’’’’’’’’ ’’’’’’’’’€ļ ,__inference_functional_1_layer_call_fn_17400¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI Š 0 ’’’’’’’’’€ )Š& 1 ’’’’’’’’’ ’’’’’’’’’€į ,__inference_functional_1_layer_call_fn_21153°¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI Š 0 ’’’’’’’’’€ )Š& 1 ’’’’’’’’’ ’’’’’’’’’€į ,__inference_functional_1_layer_call_fn_21560°¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI Š 0 ’’’’’’’’’€ )Š& 1 ’’’’’’’’’ ’’’’’’’’’€“ A__inference_lambda_layer_call_and_return_conditional_losses_29244oE¢B ;¢8 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€  p Ŗ"&¢# Š 0 ’’’’’’’’’€ š“ A__inference_lambda_layer_call_and_return_conditional_losses_29253oE¢B ;¢8 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€  p Ŗ"&¢# Š 0 ’’’’’’’’’€ šŒ &__inference_lambda_layer_call_fn_29258bE¢B ;¢8 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€  p Ŗ"Š ’’’’’’’’’€Œ &__inference_lambda_layer_call_fn_29263bE¢B ;¢8 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€  p Ŗ"Š ’’’’’’’’’€Æ K__inference_pooler_transform_layer_call_and_return_conditional_losses_29274`ķī0¢- &¢# !Š inputs ’’’’’’’’’€ Ŗ"&¢# Š 0 ’’’’’’’’’€ š‡ 0__inference_pooler_transform_layer_call_fn_29283Sķī0¢- &¢# !Š inputs ’’’’’’’’’€ Ŗ"Š ’’’’’’’’’€Č M__inference_position_embedding_layer_call_and_return_conditional_losses_25355w)=¢: 3¢0 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š  2__inference_position_embedding_layer_call_fn_25362j)=¢: 3¢0 .Š+ inputs ’’’’’’’’’ ’’’’’’’’’€ Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€ƒ N__inference_self_attention_mask_layer_call_and_return_conditional_losses_25497°q¢n g¢d bš_ 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ Ŗ";¢8 1Š. 0' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ šŪ 3__inference_self_attention_mask_layer_call_fn_25503£q¢n g¢d bš_ 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ Ŗ".Š+' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’į #__inference_signature_wrapper_17809¹¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī×¢Ó ¢ ĖŖĒ ; input_mask-Š* input_mask ’’’’’’’’’ ’’’’’’’’’ C input_type_ids1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ C input_word_ids1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’"£ŖŸ E transformer_encoder.Š+ transformer_encoder ’’’’’’’’’€ V transformer_encoder_1=Š: transformer_encoder_1 ’’’’’’’’’ ’’’’’’’’’€Ÿ N__inference_transformer/layer_0_layer_call_and_return_conditional_losses_25631Ģ-./0123456789:;<‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šŸ N__inference_transformer/layer_0_layer_call_and_return_conditional_losses_25738Ģ-./0123456789:;<‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š÷ 3__inference_transformer/layer_0_layer_call_fn_25776æ-./0123456789:;<‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€÷ 3__inference_transformer/layer_0_layer_call_fn_25814æ-./0123456789:;<‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€° O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_28741Ü ĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š° O__inference_transformer/layer_10_layer_call_and_return_conditional_losses_28848Ü ĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šˆ 4__inference_transformer/layer_10_layer_call_fn_28886Ļ ĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€ˆ 4__inference_transformer/layer_10_layer_call_fn_28924Ļ ĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€° O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_29052Ü ŻŽßąįāćäåęēčéźėģ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š° O__inference_transformer/layer_11_layer_call_and_return_conditional_losses_29159Ü ŻŽßąįāćäåęēčéźėģ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šˆ 4__inference_transformer/layer_11_layer_call_fn_29197Ļ ŻŽßąįāćäåęēčéźėģ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€ˆ 4__inference_transformer/layer_11_layer_call_fn_29235Ļ ŻŽßąįāćäåęēčéźėģ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ÿ N__inference_transformer/layer_1_layer_call_and_return_conditional_losses_25942Ģ=>?@ABCDEFGHIJKL‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šŸ N__inference_transformer/layer_1_layer_call_and_return_conditional_losses_26049Ģ=>?@ABCDEFGHIJKL‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š÷ 3__inference_transformer/layer_1_layer_call_fn_26087æ=>?@ABCDEFGHIJKL‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€÷ 3__inference_transformer/layer_1_layer_call_fn_26125æ=>?@ABCDEFGHIJKL‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ÿ N__inference_transformer/layer_2_layer_call_and_return_conditional_losses_26253ĢMNOPQRSTUVWXYZ[\‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šŸ N__inference_transformer/layer_2_layer_call_and_return_conditional_losses_26360ĢMNOPQRSTUVWXYZ[\‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š÷ 3__inference_transformer/layer_2_layer_call_fn_26398æMNOPQRSTUVWXYZ[\‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€÷ 3__inference_transformer/layer_2_layer_call_fn_26436æMNOPQRSTUVWXYZ[\‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ÿ N__inference_transformer/layer_3_layer_call_and_return_conditional_losses_26564Ģ]^_`abcdefghijkl‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šŸ N__inference_transformer/layer_3_layer_call_and_return_conditional_losses_26671Ģ]^_`abcdefghijkl‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š÷ 3__inference_transformer/layer_3_layer_call_fn_26709æ]^_`abcdefghijkl‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€÷ 3__inference_transformer/layer_3_layer_call_fn_26747æ]^_`abcdefghijkl‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ÿ N__inference_transformer/layer_4_layer_call_and_return_conditional_losses_26875Ģmnopqrstuvwxyz{|‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šŸ N__inference_transformer/layer_4_layer_call_and_return_conditional_losses_26982Ģmnopqrstuvwxyz{|‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š÷ 3__inference_transformer/layer_4_layer_call_fn_27020æmnopqrstuvwxyz{|‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€÷ 3__inference_transformer/layer_4_layer_call_fn_27058æmnopqrstuvwxyz{|‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€¬ N__inference_transformer/layer_5_layer_call_and_return_conditional_losses_27186Ł}~€‚ƒ„…†‡ˆ‰Š‹Œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š¬ N__inference_transformer/layer_5_layer_call_and_return_conditional_losses_27293Ł}~€‚ƒ„…†‡ˆ‰Š‹Œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š„ 3__inference_transformer/layer_5_layer_call_fn_27331Ģ}~€‚ƒ„…†‡ˆ‰Š‹Œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€„ 3__inference_transformer/layer_5_layer_call_fn_27369Ģ}~€‚ƒ„…†‡ˆ‰Š‹Œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Æ N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_27497Ü Ž‘’“”•–—˜™š›œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šÆ N__inference_transformer/layer_6_layer_call_and_return_conditional_losses_27604Ü Ž‘’“”•–—˜™š›œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š‡ 3__inference_transformer/layer_6_layer_call_fn_27642Ļ Ž‘’“”•–—˜™š›œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€‡ 3__inference_transformer/layer_6_layer_call_fn_27680Ļ Ž‘’“”•–—˜™š›œ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Æ N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_27808Ü žŸ ”¢£¤„¦§Ø©Ŗ«¬‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šÆ N__inference_transformer/layer_7_layer_call_and_return_conditional_losses_27915Ü žŸ ”¢£¤„¦§Ø©Ŗ«¬‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š‡ 3__inference_transformer/layer_7_layer_call_fn_27953Ļ žŸ ”¢£¤„¦§Ø©Ŗ«¬‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€‡ 3__inference_transformer/layer_7_layer_call_fn_27991Ļ žŸ ”¢£¤„¦§Ø©Ŗ«¬‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Æ N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_28119Ü ­®Æ°±²³“µ¶·ø¹ŗ»¼‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šÆ N__inference_transformer/layer_8_layer_call_and_return_conditional_losses_28226Ü ­®Æ°±²³“µ¶·ø¹ŗ»¼‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š‡ 3__inference_transformer/layer_8_layer_call_fn_28264Ļ ­®Æ°±²³“µ¶·ø¹ŗ»¼‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€‡ 3__inference_transformer/layer_8_layer_call_fn_28302Ļ ­®Æ°±²³“µ¶·ø¹ŗ»¼‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Æ N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_28430Ü ½¾æĄĮĀĆÄÅĘĒČÉŹĖĢ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ šÆ N__inference_transformer/layer_9_layer_call_and_return_conditional_losses_28537Ü ½¾æĄĮĀĆÄÅĘĒČÉŹĖĢ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š‡ 3__inference_transformer/layer_9_layer_call_fn_28575Ļ ½¾æĄĮĀĆÄÅĘĒČÉŹĖĢ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€‡ 3__inference_transformer/layer_9_layer_call_fn_28613Ļ ½¾æĄĮĀĆÄÅĘĒČÉŹĖĢ‚¢ x¢u ošl 0Š- inputs/0 ’’’’’’’’’ ’’’’’’’’’€ 8Š5 inputs/1' ’’’’’’’’’ ’’’’’’’’’ ’’’’’’’’’ p Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ÿ N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12049̶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM +Š( 0/0 ’’’’’’’’’ ’’’’’’’’’€ Š 0/1 ’’’’’’’’’€ šŸ N__inference_transformer_encoder_layer_call_and_return_conditional_losses_12475̶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM +Š( 0/0 ’’’’’’’’’ ’’’’’’’’’€ Š 0/1 ’’’’’’’’’€ š‘ N__inference_transformer_encoder_layer_call_and_return_conditional_losses_23158¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM +Š( 0/0 ’’’’’’’’’ ’’’’’’’’’€ Š 0/1 ’’’’’’’’’€ š‘ N__inference_transformer_encoder_layer_call_and_return_conditional_losses_24497¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"Z¢W PšM +Š( 0/0 ’’’’’’’’’ ’’’’’’’’’€ Š 0/1 ’’’’’’’’’€ šö 3__inference_transformer_encoder_layer_call_fn_13309¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ Š 1 ’’’’’’’’’€ö 3__inference_transformer_encoder_layer_call_fn_14142¾¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķī“¢° Ø¢¤ ™š• 1Š. input_word_ids ’’’’’’’’’ ’’’’’’’’’ -Š* input_mask ’’’’’’’’’ ’’’’’’’’’ 1Š. input_type_ids ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ Š 1 ’’’’’’’’’€č 3__inference_transformer_encoder_layer_call_fn_24904°¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ Š 1 ’’’’’’’’’€č 3__inference_transformer_encoder_layer_call_fn_25311°¶()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ”¢£¤„¦§Ø©Ŗ«¬­®Æ°±²³“µ¶·ø¹ŗ»¼½¾æĄĮĀĆÄÅĘĒČÉŹĖĢĶĪĻŠŃŅÓŌÕÖ×ŲŁŚŪÜŻŽßąįāćäåęēčéźėģķ¢ 𢖠‹š‡ +Š( inputs/0 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/1 ’’’’’’’’’ ’’’’’’’’’ +Š( inputs/2 ’’’’’’’’’ ’’’’’’’’’ p  Ŗ"LšI )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ Š 1 ’’’’’’’’’€Ą J__inference_type_embeddings_layer_call_and_return_conditional_losses_25380r*8¢5 .¢+ )Š& inputs ’’’’’’’’’ ’’’’’’’’’ Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š˜ /__inference_type_embeddings_layer_call_fn_25387e*8¢5 .¢+ )Š& inputs ’’’’’’’’’ ’’’’’’’’’ Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€Ą J__inference_word_embeddings_layer_call_and_return_conditional_losses_25325r(8¢5 .¢+ )Š& inputs ’’’’’’’’’ ’’’’’’’’’ Ŗ"3¢0 )Š& 0 ’’’’’’’’’ ’’’’’’’’’€ š˜ /__inference_word_embeddings_layer_call_fn_25332e(8¢5 .¢+ )Š& inputs ’’’’’’’’’ ’’’’’’’’’ Ŗ"&Š# ’’’’’’’’’ ’’’’’’’’’€